Sep 10 19:18:12.761248 ip-10-0-139-35 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 19:18:12.761262 ip-10-0-139-35 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 19:18:12.761272 ip-10-0-139-35 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 19:18:12.761543 ip-10-0-139-35 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 19:18:22.167975 ip-10-0-139-35 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 10 19:18:22.167986 ip-10-0-139-35 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot bce0f4e006af48d38a8027a4446d77c4 --
Sep 10 19:20:06.932239 ip-10-0-139-35 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 19:20:07.384527 ip-10-0-139-35 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:20:07.384527 ip-10-0-139-35 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 19:20:07.384527 ip-10-0-139-35 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:20:07.384527 ip-10-0-139-35 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 19:20:07.384527 ip-10-0-139-35 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:20:07.385426 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.385336    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 19:20:07.391766 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391736    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:20:07.391766 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391760    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:20:07.391766 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391764    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:20:07.391766 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391767    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:20:07.391766 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391770    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:20:07.391766 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391773    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:20:07.391766 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391776    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391779    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391782    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391785    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391788    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391791    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391793    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391798    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391803    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391806    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391809    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391812    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391815    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391817    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391820    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391823    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391826    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391829    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391831    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391834    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:20:07.392044 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391837    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391839    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391842    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391845    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391847    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391850    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391853    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391855    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391857    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391860    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391862    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391865    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391868    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391871    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391874    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391877    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391880    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391903    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391909    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:20:07.392611 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391913    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391916    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391919    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391922    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391924    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391927    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391930    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391932    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391935    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391938    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391941    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391944    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391946    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391950    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391954    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391957    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391959    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391962    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391965    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:20:07.393102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391967    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391970    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391974    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391977    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391980    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391982    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391985    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391989    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391992    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391995    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.391998    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392000    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392003    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392006    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392009    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392011    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392014    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392016    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392019    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392023    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:20:07.393561 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392025    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392028    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392030    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392033    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392035    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392038    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392041    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392043    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392047    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392050    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392052    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392055    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392058    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392061    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392063    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392776    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392783    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392786    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392789    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392792    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:20:07.394063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392795    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392798    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392800    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392804    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392808    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392812    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392816    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392819    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392821    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392824    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392827    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392830    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392833    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392836    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392838    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392842    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392844    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392847    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392850    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392852    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:20:07.394543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392855    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392857    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392860    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392863    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392866    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392868    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392871    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392873    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392876    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392879    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392881    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392900    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392904    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392907    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392909    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392912    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392915    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392917    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392920    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392923    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:20:07.395063 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392925    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392928    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392930    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392933    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392935    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392939    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392942    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392945    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392947    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392950    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392952    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392955    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392957    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392960    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392962    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392967    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392971    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392974    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392977    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:20:07.395553 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392980    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392982    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392985    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392988    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392991    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392993    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392996    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.392998    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393001    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393003    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393006    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393008    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393012    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393014    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393017    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393019    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393022    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393024    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393031    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393034    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:20:07.396053 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393037    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393040    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393042    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393047    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393050    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393053    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393056    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393059    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393062    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393064    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393067    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393069    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393072    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393075    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393077    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393466    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393470    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393474    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393476    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:20:07.396547 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393479    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393482    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393484    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393487    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393489    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393492    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393495    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393498    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393500    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393503    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393506    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393511    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393533    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393536    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393539    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393542    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393545    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393547    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393550    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:20:07.397038 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393553    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393555    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393558    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393561    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393564    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393566    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393569    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393572    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393575    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393578    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393580    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393583    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393585    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393588    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393590    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393593    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393596    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393598    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393601    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393603    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:20:07.397533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393606    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393608    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393611    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393613    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393618    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393620    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393623    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393626    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393629    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393631    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393634    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393636    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393639    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393641    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393644    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393647    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393649    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393652    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393660    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:20:07.398035 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393663    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393665    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393668    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393671    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393675    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393679    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393682    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393685    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393691    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393694    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393697    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393700    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393702    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393705    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393708    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393710    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393713    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393717    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393719    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:20:07.398529 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393722    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393725    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393728    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393731    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393733    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393736    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393739    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393741    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393744    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393746    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393748    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393751    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393754    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393756    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393759    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393761    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393764    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.393767    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393845    2562 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393878    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393899    2562 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 19:20:07.399019 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393906    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393911    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393915    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393920    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393924    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393927    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393930    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393934    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393939    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393944    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393947    2562 flags.go:64] FLAG: --cgroup-root=""
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393950    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393953    2562 flags.go:64] FLAG: --client-ca-file=""
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393958    2562 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393961    2562 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393965    2562 flags.go:64] FLAG: --cluster-domain=""
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393968    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393971    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393975    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393978    2562 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393982    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393985    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393988    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 19:20:07.399522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393991    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393994    2562 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.393997    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394000    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394003    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394006    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394011    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394014    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394017    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394020    2562 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394022    2562 flags.go:64] FLAG: --enable-server="true"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394025    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394029    2562 flags.go:64] FLAG: --event-burst="100"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394032    2562 flags.go:64] FLAG: --event-qps="50"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394035    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394038    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394041    2562 flags.go:64] FLAG: --eviction-hard=""
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394046    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394051    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394053    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394056    2562 flags.go:64] FLAG: --eviction-soft=""
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394060    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394062    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394066    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394069    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 19:20:07.400082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394073    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394076    2562 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394079    2562 flags.go:64] FLAG: --feature-gates=""
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394082    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394085    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394088    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394091    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394094    2562 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394097    2562 flags.go:64] FLAG: --help="false"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394100    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394103    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394106    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394109    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394112    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394116    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394118    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394121    2562 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394124    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394127    2562 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394130    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394133    2562 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394137    2562 flags.go:64] FLAG: --kube-reserved=""
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394139    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394142    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 19:20:07.400715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394146    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394150    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394153    2562 flags.go:64] FLAG: --lock-file=""
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394156    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394159    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394162    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394167    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394170    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394173    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394176    2562 flags.go:64] FLAG: --logging-format="text"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394179    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394182    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394185    2562 flags.go:64] FLAG: --manifest-url=""
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394188    2562 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394192    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394195    2562 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394199    2562 flags.go:64] FLAG: --max-pods="110"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394202    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394205    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394208    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394211    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394214    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394217    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394220    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394228    2562 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 19:20:07.401390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394231    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394233    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394236    2562 flags.go:64] FLAG: --pod-cidr=""
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394239    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394245    2562 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394248    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394250    2562 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394255    2562 flags.go:64] FLAG: --port="10250"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394258    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394261    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d0141155d3941fcc"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394264    2562 flags.go:64] FLAG: --qos-reserved=""
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394268    2562 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394271    2562 flags.go:64] FLAG: --register-node="true"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394274    2562 flags.go:64] FLAG: --register-with-taints=""
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394277    2562 flags.go:64] FLAG: --registry-burst="10"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394281    2562 flags.go:64] FLAG: --registry-qps="5"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394283    2562 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394286    2562 flags.go:64] FLAG: --reserved-memory=""
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394290    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394293    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394296    2562 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394299    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394302    2562 flags.go:64] FLAG: --runonce="false"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394309    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394312    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 19:20:07.402008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394316    2562 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394319    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394321    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394325    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394328    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394331    2562 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394334    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394337    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394339    2562 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394343    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394346    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394349    2562 flags.go:64] FLAG: --system-cgroups=""
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394352    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394357    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394361    2562 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394364    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394369    2562 flags.go:64] FLAG: --tls-min-version=""
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394371    2562 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394374    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394377    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394380    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394383    2562 flags.go:64] FLAG: --v="2"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394387    2562 flags.go:64] FLAG: --version="false"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394391    2562 flags.go:64] FLAG: --vmodule=""
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394395    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 19:20:07.402655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.394398    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394516    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394520    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394523    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394526    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394529    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394531    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394533    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394536    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394538    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394541    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394544    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394547    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394550    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394552    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394555    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394558    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394560    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394563    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:20:07.403298 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394565    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394568    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394572    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394577    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394580    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394584    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394587    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394589    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394592    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394595    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394597    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394600    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394603    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394605    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394608    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394610    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394613    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394615    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394618    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:20:07.403767 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394620    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394622    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394625    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394627    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394630    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394633    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394635    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394638    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394640    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394643    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394646    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394648    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394651    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394653    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394655    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394659    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394662    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394666    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394668    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394671    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:20:07.404259 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394673    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394676    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394680    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394683    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394686    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394689    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394693    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394695    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394698    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394701    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394703    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394706    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394708    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394711    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394713    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394716    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394719    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394721    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394723    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394726    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:20:07.404752 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394728    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394731    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394734    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394737    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394740    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394742    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394745    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394749    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394751    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394755    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394758    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394760    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394763    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394766    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394768    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394771    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394774    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394776    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394779    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394782    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:20:07.405315 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394784    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.394788    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.395817    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.403721    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.403741    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.403757    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.403764    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403821    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403827    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403830    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403833    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403836    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403840    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403843    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:20:07.405797 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403846    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403849    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403851    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403854    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403856    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403859    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403862    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403865    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403869    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403873    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403876    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403879    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403882    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403898    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403901    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403903    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403906    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403909    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403912    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403914    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:20:07.406168 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403917    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403919    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403924    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403927    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403931    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403935    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403938    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403941    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403944    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403946    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403949    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403952    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403954    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403957    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403960    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403963    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403965    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403968    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403970    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403973    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:20:07.406657 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403976    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403979    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403982    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403984    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403987    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403989    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403992    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403996    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.403999    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404002    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404005    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404008    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404010    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404013    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404017    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404020    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404023    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404025    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404028    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:20:07.407169 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404031    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404033    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404036    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404039    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404041    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404044    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404046    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404049    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404051    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404054    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404056    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404059    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404061    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404064    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404066    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404069    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404071    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404074    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404077    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404079    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:20:07.407632 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404082    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404084    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404087    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404089    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404092    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404094    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404097    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404100    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404102    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404105    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404108    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404111    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404113    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.404118    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404246    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:20:07.408124 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404251    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404254    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404257    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404260    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404263    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404266    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404269    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404272    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404275    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404278    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404281    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404284    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404287    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404289    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404292    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404295    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404298    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404300    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404303    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404305    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:20:07.408498 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404308    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404310    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404313    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404316    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404319    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404321    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404324    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404327    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404330    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404332    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404334    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404337    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404339    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404342    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404344    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404347    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404349    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404352    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404354    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404358    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:20:07.409064 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404361    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404364    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404367    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404370    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404372    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404375    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404377    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404380    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404383    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404385    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404388    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404391    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404393    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404395    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404398    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404401    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404403    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404407    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404410    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:20:07.409579 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404413    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404417    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404419    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404422    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404425    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404427    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404430    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404432    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404435    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404437    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404440    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404442    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404445    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404447    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404450    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404452    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404455    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404458    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404460    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404463    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:20:07.410061 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404465    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404468    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404470    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404473    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404475    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404478    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404480    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404483    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404485    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404488    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404490    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404493    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404495    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404498    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404503    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404505    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404508    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404510    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:20:07.410550 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:07.404513    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:20:07.411005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.404518    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:20:07.411005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.404639    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 19:20:07.411005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.407016    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 19:20:07.411005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.408102    2562 server.go:1019] "Starting client certificate rotation"
Sep 10 19:20:07.411005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.408203    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:20:07.411005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.408241    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:20:07.411728 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.411710    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:20:07.417332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.417305    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:20:07.425523 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.425500    2562 log.go:25] "Validated CRI v1 runtime API"
Sep 10 19:20:07.428869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.428854    2562 log.go:25] "Validated CRI v1 image API"
Sep 10 19:20:07.431944 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.431926    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 19:20:07.435278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.435256    2562 fs.go:135] Filesystem UUIDs: map[49ad76a7-4c06-4a60-befa-3f4b3cd7ed52:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f56a1a1d-66a5-4bc9-9a75-fca0321dc38f:/dev/nvme0n1p3]
Sep 10 19:20:07.435353 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.435277    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 19:20:07.440522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.440392    2562 manager.go:216] Machine: {Timestamp:2026-09-10 19:20:07.438966812 +0000 UTC m=+0.379000635 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104789 MemoryCapacity:33164468224 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29c8a2a1f25244614e7381a70cd78a SystemUUID:ec29c8a2-a1f2-5244-614e-7381a70cd78a BootID:bce0f4e0-06af-48d3-8a80-27a4446d77c4 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:70:fa:c4:5b:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:70:fa:c4:5b:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:67:44:d9:91:5c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164468224 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 19:20:07.440522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.440509    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 19:20:07.440638 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.440605    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 19:20:07.441858 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.441840    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:20:07.444092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.444060    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 19:20:07.444314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.444101    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-35.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 19:20:07.445722 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.445710    2562 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 19:20:07.445777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.445731    2562 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 19:20:07.445777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.445743    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:20:07.445777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.445763    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:20:07.447663 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.447652    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 19:20:07.452427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.452413    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:20:07.452569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.452558    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 19:20:07.452628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.452597    2562 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 19:20:07.452628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.452613    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 19:20:07.452716 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.452633    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 19:20:07.452716 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.452651    2562 kubelet.go:398] "Adding apiserver pod source"
Sep 10 19:20:07.452716 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.452664    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 19:20:07.453077 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.453062    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:20:07.453153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.453087    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:20:07.455516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.455492    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 19:20:07.459770 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.459754    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 19:20:07.459850 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.459786    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 19:20:07.461015 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.460060    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 19:20:07.461073 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461063    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 19:20:07.461112 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461076    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 19:20:07.461112 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461086    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461126    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461134    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461148    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461154    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461162    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461168    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461177    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 19:20:07.461198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.461182    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 19:20:07.462051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.462039    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 19:20:07.462051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.462050    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 19:20:07.463600 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.463576    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 19:20:07.463677 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.463576    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 19:20:07.466312 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.466299    2562 server.go:1268] "Started kubelet"
Sep 10 19:20:07.466378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.466355    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 19:20:07.466821 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.466772    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 19:20:07.466878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.466844    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 19:20:07.467212 ip-10-0-139-35 systemd[1]: Started Kubernetes Kubelet.
Sep 10 19:20:07.468451 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.468433    2562 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 19:20:07.473443 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.473423    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 19:20:07.475193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.475169    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-35.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 19:20:07.476411 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.475258    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-35.ec2.internal.18d40bdfb80af3cc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-35.ec2.internal,UID:ip-10-0-139-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-35.ec2.internal,},FirstTimestamp:2026-09-10 19:20:07.466267596 +0000 UTC m=+0.406301420,LastTimestamp:2026-09-10 19:20:07.466267596 +0000 UTC m=+0.406301420,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-35.ec2.internal,}"
Sep 10 19:20:07.492824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.492795    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r2svk"
Sep 10 19:20:07.500670 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.500646    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r2svk"
Sep 10 19:20:07.606933 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.606734    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 19:20:07.611671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.611652    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 19:20:07.611767 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.611668    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 19:20:07.611816 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.611788    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:07.612537 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.612515    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 19:20:07.612537 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.612517    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 19:20:07.612733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.612546    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 19:20:07.612733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.612676    2562 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 19:20:07.612733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.612686    2562 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 19:20:07.613769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.613752    2562 factory.go:153] Registering CRI-O factory
Sep 10 19:20:07.613858 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.613807    2562 factory.go:223] Registration of the crio container factory successfully
Sep 10 19:20:07.613973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.613951    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 19:20:07.613973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.613967    2562 factory.go:55] Registering systemd factory
Sep 10 19:20:07.613973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.613977    2562 factory.go:223] Registration of the systemd container factory successfully
Sep 10 19:20:07.614157 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.614030    2562 factory.go:103] Registering Raw factory
Sep 10 19:20:07.614157 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.614075    2562 manager.go:1196] Started watching for new ooms in manager
Sep 10 19:20:07.614364 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.614338    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 19:20:07.614551 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.614515    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 19:20:07.614707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.614694    2562 manager.go:349] Starting recovery of all containers
Sep 10 19:20:07.626268 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.626242    2562 manager.go:354] Recovery completed
Sep 10 19:20:07.630552 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.630539    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:20:07.632485 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.632468    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:20:07.632545 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.632499    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:20:07.632545 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.632509    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:20:07.633087 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.633074    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 19:20:07.633152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.633087    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 19:20:07.633152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.633093    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:20:07.633152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.633106    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:20:07.633758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.633743    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:20:07.633796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.633769    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:20:07.633796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.633779    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:20:07.635314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.635262    2562 policy_none.go:49] "None policy: Start"
Sep 10 19:20:07.635314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.635277    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 19:20:07.635314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.635287    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 19:20:07.637265 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.637253    2562 policy_none.go:47] "Start"
Sep 10 19:20:07.682604 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.682582    2562 manager.go:337] "Starting Device Plugin manager"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.682630    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.682645    2562 server.go:87] "Starting device plugin registration server"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.683107    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.683122    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.683320    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.683404    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.683412    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.683909    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.683952    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.687534    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.688791    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.688832    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.688853    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.688918    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 19:20:07.691570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.690577    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:20:07.783873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.783828    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:20:07.784980 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.784960    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:20:07.785126 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.784992    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:20:07.785126 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.785006    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:20:07.785126 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.785029    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.789102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.789083    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal"]
Sep 10 19:20:07.789172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.789164    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:20:07.790055 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.790037    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:20:07.790125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.790068    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:20:07.790125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.790079    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:20:07.792494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.792482    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:20:07.792623 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.792608    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.792673 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.792643    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:20:07.793200 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.793185    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:20:07.793283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.793189    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:20:07.793283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.793233    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:20:07.793283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.793249    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:20:07.793283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.793208    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:20:07.793283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.793277    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:20:07.793921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.793908    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.793979 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.793926    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-35.ec2.internal\": node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:07.795543 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.795528    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.795597 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.795562    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:20:07.796270 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.796251    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:20:07.796368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.796278    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:20:07.796368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.796291    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:20:07.814649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.814613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dd066b74f4ff1cd5196e5b51bb3dd9d4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal\" (UID: \"dd066b74f4ff1cd5196e5b51bb3dd9d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.814793 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.814664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd066b74f4ff1cd5196e5b51bb3dd9d4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal\" (UID: \"dd066b74f4ff1cd5196e5b51bb3dd9d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.814793 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.814732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50f4e7fca0ff25f9b39860d2874f4285-config\") pod \"kube-apiserver-proxy-ip-10-0-139-35.ec2.internal\" (UID: \"50f4e7fca0ff25f9b39860d2874f4285\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.822709 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.822685    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:07.825752 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.825736    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-35.ec2.internal\" not found" node="ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.830242 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.830225    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-35.ec2.internal\" not found" node="ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.915277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.915184    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dd066b74f4ff1cd5196e5b51bb3dd9d4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal\" (UID: \"dd066b74f4ff1cd5196e5b51bb3dd9d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.915277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.915220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd066b74f4ff1cd5196e5b51bb3dd9d4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal\" (UID: \"dd066b74f4ff1cd5196e5b51bb3dd9d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.915277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.915239    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50f4e7fca0ff25f9b39860d2874f4285-config\") pod \"kube-apiserver-proxy-ip-10-0-139-35.ec2.internal\" (UID: \"50f4e7fca0ff25f9b39860d2874f4285\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.915494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.915281    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50f4e7fca0ff25f9b39860d2874f4285-config\") pod \"kube-apiserver-proxy-ip-10-0-139-35.ec2.internal\" (UID: \"50f4e7fca0ff25f9b39860d2874f4285\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.915494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.915301    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dd066b74f4ff1cd5196e5b51bb3dd9d4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal\" (UID: \"dd066b74f4ff1cd5196e5b51bb3dd9d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.915494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:07.915308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd066b74f4ff1cd5196e5b51bb3dd9d4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal\" (UID: \"dd066b74f4ff1cd5196e5b51bb3dd9d4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:07.923304 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:07.923279    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:08.023932 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:08.023874    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:08.124352 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:08.124296    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:08.129899 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.129873    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:08.134172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.134154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:08.225496 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:08.225397    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:08.326524 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:08.326496    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:08.413277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.413241    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 19:20:08.413773 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.413421    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:20:08.427440 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:08.427409    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:08.527724 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:08.527612    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-35.ec2.internal\" not found"
Sep 10 19:20:08.603050 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.603018    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:20:08.612495 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.612461    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 19:20:08.612758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.612461    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:08.613051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.613005    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 19:15:07 +0000 UTC" deadline="2028-04-30 21:46:51.283334964 +0000 UTC"
Sep 10 19:20:08.613051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.613048    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14354h26m42.670290746s"
Sep 10 19:20:08.630251 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.630225    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:20:08.636508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.636480    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:20:08.655704 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.655673    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:20:08.656886 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.656868    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal"
Sep 10 19:20:08.670240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.670216    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:20:08.679875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.679846    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wmdb"
Sep 10 19:20:08.687293 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.687272    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wmdb"
Sep 10 19:20:08.768648 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:08.768609    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f4e7fca0ff25f9b39860d2874f4285.slice/crio-565a9b6ef931860bfeeb8e97e45124148775aeaa47d2b3392b62a50d51720446 WatchSource:0}: Error finding container 565a9b6ef931860bfeeb8e97e45124148775aeaa47d2b3392b62a50d51720446: Status 404 returned error can't find the container with id 565a9b6ef931860bfeeb8e97e45124148775aeaa47d2b3392b62a50d51720446
Sep 10 19:20:08.768907 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:08.768863    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd066b74f4ff1cd5196e5b51bb3dd9d4.slice/crio-6f606d5af8bcd6914e9afda80ed0ded45221379443d549b62b1459538a0ee96a WatchSource:0}: Error finding container 6f606d5af8bcd6914e9afda80ed0ded45221379443d549b62b1459538a0ee96a: Status 404 returned error can't find the container with id 6f606d5af8bcd6914e9afda80ed0ded45221379443d549b62b1459538a0ee96a
Sep 10 19:20:08.772836 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:08.772821    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:20:09.155733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.155699    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:20:09.409624 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.409539    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:20:09.455675 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.455640    2562 apiserver.go:52] "Watching apiserver"
Sep 10 19:20:09.464878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.464852    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 19:20:09.465329 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.465301    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-xfq2p","openshift-dns/node-resolver-2htct","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal","openshift-multus/multus-additional-cni-plugins-mm5gp","openshift-multus/multus-kq4fw","kube-system/global-pull-secret-syncer-nhl6m","openshift-image-registry/node-ca-bhgv7","openshift-multus/network-metrics-daemon-slrvh","openshift-network-diagnostics/network-check-target-xgn2n","openshift-network-operator/iptables-alerter-6pr6j","openshift-ovn-kubernetes/ovnkube-node-wk98n","kube-system/konnectivity-agent-7rxpl","kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"]
Sep 10 19:20:09.468083 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.468054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.471176 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.471138    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nv5lh\""
Sep 10 19:20:09.471290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.471187    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.471533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.471511    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 19:20:09.471796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.471777    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 19:20:09.471895 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.471804    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 19:20:09.473546 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.473523    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.474569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.474501    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 19:20:09.474569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.474553    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 19:20:09.475202 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.475177    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xmln5\""
Sep 10 19:20:09.475348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.475327    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 19:20:09.478027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.476041    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 19:20:09.478027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.476265    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 19:20:09.478027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.476330    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 19:20:09.478027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.477034    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 19:20:09.478027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.477177    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 19:20:09.478027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.477454    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.480602 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.478583    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 19:20:09.480602 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.479155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rdwrv\""
Sep 10 19:20:09.481975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.481017    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2g28\""
Sep 10 19:20:09.481975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.481336    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 19:20:09.481975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.481850    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.481975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.481904    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 19:20:09.482241 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.481980    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 19:20:09.482241 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.482148    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 19:20:09.482537 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.482460    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 19:20:09.484424 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.484402    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.484527 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.484501    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:09.485052 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.485030    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-th4lz\""
Sep 10 19:20:09.485327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.485306    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 19:20:09.487332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.487311    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:09.487455 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.487431    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:09.489701 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.489680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:09.489814 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.489781    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:09.492898 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.492874    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.495933 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.495800    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.496338 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.496319    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 19:20:09.496770 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.496750    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-phmnk\""
Sep 10 19:20:09.496871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.496853    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 19:20:09.499413 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.499388    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:09.499510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.499441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 19:20:09.499873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.499741    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:09.499873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.499782    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fqwr5\""
Sep 10 19:20:09.500418 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.500239    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.502899 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.502877    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:09.504454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.504427    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:09.504551 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.504477    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:09.504760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.504725    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-79rnn\""
Sep 10 19:20:09.505881 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.505863    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-46q5h\""
Sep 10 19:20:09.506370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.506351    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 19:20:09.506455 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.506410    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 19:20:09.514248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.514224    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 19:20:09.528190 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528159    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-sys\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.528313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.528313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528245    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-systemd-units\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.528313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-ovn\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.528313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528296    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-cni-bin\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.528533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528329    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-socket-dir-parent\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.528533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c71f7e5-908d-4f2d-b087-2a5adb55ae5b-agent-certs\") pod \"konnectivity-agent-7rxpl\" (UID: \"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b\") " pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:09.528533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-log-socket\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.528533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528450    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.528533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-tuned\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.528533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528505    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5xn7\" (UniqueName: \"kubernetes.io/projected/5147a1e1-114c-4f51-b475-3a681c741cb4-kube-api-access-l5xn7\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-device-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aeec9d3-ec71-422b-83de-777f2a89e902-cni-binary-copy\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-multus-certs\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-lib-modules\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528726    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5147a1e1-114c-4f51-b475-3a681c741cb4-tmp\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528751    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-node-log\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovnkube-script-lib\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.528846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528826    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-cni-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g4wz\" (UniqueName: \"kubernetes.io/projected/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-kube-api-access-2g4wz\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-socket-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysconfig\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-conf-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.528995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491040f7-5239-4e74-ac39-1b5a8190e333-tmp-dir\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529025    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz9mr\" (UniqueName: \"kubernetes.io/projected/491040f7-5239-4e74-ac39-1b5a8190e333-kube-api-access-cz9mr\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529077    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysctl-d\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529111    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-run-netns\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjdts\" (UniqueName: \"kubernetes.io/projected/6165147e-6bbf-4e85-b4f3-28b309b0004a-kube-api-access-kjdts\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529172    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-cnibin\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-netns\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zln2l\" (UniqueName: \"kubernetes.io/projected/5aeec9d3-ec71-422b-83de-777f2a89e902-kube-api-access-zln2l\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c71f7e5-908d-4f2d-b087-2a5adb55ae5b-konnectivity-ca\") pod \"konnectivity-agent-7rxpl\" (UID: \"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b\") " pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv5bj\" (UniqueName: \"kubernetes.io/projected/05dde958-85c8-4751-87d7-cd7b3d9314cc-kube-api-access-hv5bj\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.529327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-host\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6nwl\" (UniqueName: \"kubernetes.io/projected/2b3d75eb-156b-48b4-b39c-68e9907810a5-kube-api-access-n6nwl\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-etc-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529467    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovnkube-config\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-sys-fs\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529583    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-kubelet\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-etc-selinux\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-system-cni-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-iptables-alerter-script\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-host-slash\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-cnibin\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-kubelet\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.530243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-run\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-system-cni-dir\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-modprobe-d\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.529998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-kubernetes\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530036    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-systemd\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530062    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttgr6\" (UniqueName: \"kubernetes.io/projected/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-kube-api-access-ttgr6\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-etc-kubernetes\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530182    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05dde958-85c8-4751-87d7-cd7b3d9314cc-host\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-cni-netd\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530239    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-k8s-cni-cncf-io\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530264    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-daemon-config\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05dde958-85c8-4751-87d7-cd7b3d9314cc-serviceca\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530322    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-var-lib-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530354    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-env-overrides\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530384    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovn-node-metrics-cert\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-registration-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysctl-conf\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-var-lib-kubelet\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-dbus\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-cni-bin\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-cni-multus\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-hostroot\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-kubelet-config\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78hnd\" (UniqueName: \"kubernetes.io/projected/f3338777-3059-4b20-b06a-134279c5ad91-kube-api-access-78hnd\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-slash\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/491040f7-5239-4e74-ac39-1b5a8190e333-hosts-file\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.531914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.530881    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-systemd\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.632175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-device-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aeec9d3-ec71-422b-83de-777f2a89e902-cni-binary-copy\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-multus-certs\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632233    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-lib-modules\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5147a1e1-114c-4f51-b475-3a681c741cb4-tmp\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-node-log\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-device-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovnkube-script-lib\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-cni-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-multus-certs\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632373    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g4wz\" (UniqueName: \"kubernetes.io/projected/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-kube-api-access-2g4wz\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-socket-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysconfig\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-conf-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491040f7-5239-4e74-ac39-1b5a8190e333-tmp-dir\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632556    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz9mr\" (UniqueName: \"kubernetes.io/projected/491040f7-5239-4e74-ac39-1b5a8190e333-kube-api-access-cz9mr\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632581    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysctl-d\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-run-netns\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjdts\" (UniqueName: \"kubernetes.io/projected/6165147e-6bbf-4e85-b4f3-28b309b0004a-kube-api-access-kjdts\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-cnibin\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-netns\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-cni-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zln2l\" (UniqueName: \"kubernetes.io/projected/5aeec9d3-ec71-422b-83de-777f2a89e902-kube-api-access-zln2l\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c71f7e5-908d-4f2d-b087-2a5adb55ae5b-konnectivity-ca\") pod \"konnectivity-agent-7rxpl\" (UID: \"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b\") " pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:09.632830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv5bj\" (UniqueName: \"kubernetes.io/projected/05dde958-85c8-4751-87d7-cd7b3d9314cc-kube-api-access-hv5bj\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5aeec9d3-ec71-422b-83de-777f2a89e902-cni-binary-copy\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-host\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632975    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-lib-modules\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.632995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6nwl\" (UniqueName: \"kubernetes.io/projected/2b3d75eb-156b-48b4-b39c-68e9907810a5-kube-api-access-n6nwl\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633025    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-host\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-etc-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-etc-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovnkube-config\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-sys-fs\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633151    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-kubelet\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-etc-selinux\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-system-cni-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.633642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysctl-d\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633260    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-iptables-alerter-script\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633288    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-host-slash\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-run-netns\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633361    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-cnibin\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-kubelet\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-run\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633591    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-run\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-cnibin\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633580    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovnkube-config\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovnkube-script-lib\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-sys-fs\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-netns\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-node-log\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633888    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysconfig\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.633927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-socket-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.634520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634057    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-kubelet\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c71f7e5-908d-4f2d-b087-2a5adb55ae5b-konnectivity-ca\") pod \"konnectivity-agent-7rxpl\" (UID: \"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b\") " pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/491040f7-5239-4e74-ac39-1b5a8190e333-tmp-dir\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-system-cni-dir\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634580    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-etc-selinux\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-modprobe-d\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-conf-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-kubernetes\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-systemd\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-host-slash\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634667    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-system-cni-dir\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-cnibin\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgr6\" (UniqueName: \"kubernetes.io/projected/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-kube-api-access-ttgr6\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-etc-kubernetes\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.635461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-kubelet\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05dde958-85c8-4751-87d7-cd7b3d9314cc-host\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-cni-netd\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-k8s-cni-cncf-io\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-daemon-config\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634872    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05dde958-85c8-4751-87d7-cd7b3d9314cc-serviceca\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-var-lib-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-env-overrides\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634951    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovn-node-metrics-cert\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.634998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-registration-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysctl-conf\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-var-lib-kubelet\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-dbus\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-cni-bin\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-cni-multus\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-hostroot\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-kubelet-config\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.636319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635232    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-iptables-alerter-script\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635226    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635280    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78hnd\" (UniqueName: \"kubernetes.io/projected/f3338777-3059-4b20-b06a-134279c5ad91-kube-api-access-78hnd\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635308    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-slash\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.635334    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.635454    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret podName:e50d4c91-53ee-4496-9854-3a3abb0a1ca7 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:10.135408557 +0000 UTC m=+3.075442387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret") pod "global-pull-secret-syncer-nhl6m" (UID: "e50d4c91-53ee-4496-9854-3a3abb0a1ca7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b3d75eb-156b-48b4-b39c-68e9907810a5-system-cni-dir\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-var-lib-openvswitch\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635707    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-modprobe-d\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-slash\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-cni-netd\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635769    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-kubernetes\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-etc-kubernetes\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05dde958-85c8-4751-87d7-cd7b3d9314cc-host\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635850    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-systemd\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.637193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635852    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-cni-bin\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-var-lib-kubelet\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635984    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-sysctl-conf\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-dbus\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-registration-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-run-k8s-cni-cncf-io\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-env-overrides\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-kubelet-config\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.635336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/491040f7-5239-4e74-ac39-1b5a8190e333-hosts-file\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-hostroot\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636241    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-systemd\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-sys\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636287    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-systemd\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636295    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-host-var-lib-cni-multus\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636322    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-systemd-units\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.638108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/491040f7-5239-4e74-ac39-1b5a8190e333-hosts-file\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5147a1e1-114c-4f51-b475-3a681c741cb4-sys\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-ovn\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-cni-bin\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-host-cni-bin\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-systemd-units\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636484    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-run-ovn\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-socket-dir-parent\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c71f7e5-908d-4f2d-b087-2a5adb55ae5b-agent-certs\") pod \"konnectivity-agent-7rxpl\" (UID: \"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b\") " pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-log-socket\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-tuned\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5xn7\" (UniqueName: \"kubernetes.io/projected/5147a1e1-114c-4f51-b475-3a681c741cb4-kube-api-access-l5xn7\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636915    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-daemon-config\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636921    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636940    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2b3d75eb-156b-48b4-b39c-68e9907810a5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.636999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5aeec9d3-ec71-422b-83de-777f2a89e902-multus-socket-dir-parent\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.639228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.637042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6165147e-6bbf-4e85-b4f3-28b309b0004a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.640523 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.637083    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-log-socket\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.640523 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.637093    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:09.640523 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.637140    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs podName:f3338777-3059-4b20-b06a-134279c5ad91 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:10.137124908 +0000 UTC m=+3.077158736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs") pod "network-metrics-daemon-slrvh" (UID: "f3338777-3059-4b20-b06a-134279c5ad91") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:09.640523 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.637724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05dde958-85c8-4751-87d7-cd7b3d9314cc-serviceca\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.640523 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.640241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5147a1e1-114c-4f51-b475-3a681c741cb4-tmp\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.641026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.641005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-ovn-node-metrics-cert\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.641175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.641151    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c71f7e5-908d-4f2d-b087-2a5adb55ae5b-agent-certs\") pod \"konnectivity-agent-7rxpl\" (UID: \"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b\") " pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:09.643242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.643222    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5147a1e1-114c-4f51-b475-3a681c741cb4-etc-tuned\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.647145 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.646316    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:20:09.647145 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.646340    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:20:09.647145 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.646354    2562 projected.go:196] Error preparing data for projected volume kube-api-access-krw9g for pod openshift-network-diagnostics/network-check-target-xgn2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:09.647145 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:09.646415    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g podName:8a327e33-4bc2-47ae-b268-c8225e73f304 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:10.146398146 +0000 UTC m=+3.086431963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-krw9g" (UniqueName: "kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g") pod "network-check-target-xgn2n" (UID: "8a327e33-4bc2-47ae-b268-c8225e73f304") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:09.649238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.648617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv5bj\" (UniqueName: \"kubernetes.io/projected/05dde958-85c8-4751-87d7-cd7b3d9314cc-kube-api-access-hv5bj\") pod \"node-ca-bhgv7\" (UID: \"05dde958-85c8-4751-87d7-cd7b3d9314cc\") " pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.650358 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.650035    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g4wz\" (UniqueName: \"kubernetes.io/projected/6f2c3f98-09c4-4ecc-b015-21869f5ddb4b-kube-api-access-2g4wz\") pod \"iptables-alerter-6pr6j\" (UID: \"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b\") " pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.650358 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.650270    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zln2l\" (UniqueName: \"kubernetes.io/projected/5aeec9d3-ec71-422b-83de-777f2a89e902-kube-api-access-zln2l\") pod \"multus-kq4fw\" (UID: \"5aeec9d3-ec71-422b-83de-777f2a89e902\") " pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.650358 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.650304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6nwl\" (UniqueName: \"kubernetes.io/projected/2b3d75eb-156b-48b4-b39c-68e9907810a5-kube-api-access-n6nwl\") pod \"multus-additional-cni-plugins-mm5gp\" (UID: \"2b3d75eb-156b-48b4-b39c-68e9907810a5\") " pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.650901 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.650787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78hnd\" (UniqueName: \"kubernetes.io/projected/f3338777-3059-4b20-b06a-134279c5ad91-kube-api-access-78hnd\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:09.650901 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.650877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjdts\" (UniqueName: \"kubernetes.io/projected/6165147e-6bbf-4e85-b4f3-28b309b0004a-kube-api-access-kjdts\") pod \"aws-ebs-csi-driver-node-4g4b7\" (UID: \"6165147e-6bbf-4e85-b4f3-28b309b0004a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.651340 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.651315    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz9mr\" (UniqueName: \"kubernetes.io/projected/491040f7-5239-4e74-ac39-1b5a8190e333-kube-api-access-cz9mr\") pod \"node-resolver-2htct\" (UID: \"491040f7-5239-4e74-ac39-1b5a8190e333\") " pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.651496 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.651463    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgr6\" (UniqueName: \"kubernetes.io/projected/3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2-kube-api-access-ttgr6\") pod \"ovnkube-node-wk98n\" (UID: \"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.652751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.652728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5xn7\" (UniqueName: \"kubernetes.io/projected/5147a1e1-114c-4f51-b475-3a681c741cb4-kube-api-access-l5xn7\") pod \"tuned-xfq2p\" (UID: \"5147a1e1-114c-4f51-b475-3a681c741cb4\") " pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.691068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.690959    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:15:09 +0000 UTC" deadline="2028-02-18 21:52:12.27348495 +0000 UTC"
Sep 10 19:20:09.691068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.691015    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12626h32m2.582475122s"
Sep 10 19:20:09.694331 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.694276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal" event={"ID":"50f4e7fca0ff25f9b39860d2874f4285","Type":"ContainerStarted","Data":"565a9b6ef931860bfeeb8e97e45124148775aeaa47d2b3392b62a50d51720446"}
Sep 10 19:20:09.695543 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.695510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal" event={"ID":"dd066b74f4ff1cd5196e5b51bb3dd9d4","Type":"ContainerStarted","Data":"6f606d5af8bcd6914e9afda80ed0ded45221379443d549b62b1459538a0ee96a"}
Sep 10 19:20:09.782153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.782107    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bhgv7"
Sep 10 19:20:09.794478 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.794451    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:09.808908 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.808701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7"
Sep 10 19:20:09.814226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.814199    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mm5gp"
Sep 10 19:20:09.823394 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.823365    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kq4fw"
Sep 10 19:20:09.834564 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.834534    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2htct"
Sep 10 19:20:09.844572 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.844541    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6pr6j"
Sep 10 19:20:09.855870 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.855670    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xfq2p"
Sep 10 19:20:09.861984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:09.861956    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:10.142205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:10.142107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:10.142387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:10.142217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:10.142387 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.142289    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:10.142387 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.142333    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:10.142387 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.142383    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs podName:f3338777-3059-4b20-b06a-134279c5ad91 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:11.142359345 +0000 UTC m=+4.082393176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs") pod "network-metrics-daemon-slrvh" (UID: "f3338777-3059-4b20-b06a-134279c5ad91") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:10.142610 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.142405    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret podName:e50d4c91-53ee-4496-9854-3a3abb0a1ca7 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:11.142395473 +0000 UTC m=+4.082429286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret") pod "global-pull-secret-syncer-nhl6m" (UID: "e50d4c91-53ee-4496-9854-3a3abb0a1ca7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:10.243164 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:10.243128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:10.243367 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.243352    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:20:10.243431 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.243378    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:20:10.243431 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.243392    2562 projected.go:196] Error preparing data for projected volume kube-api-access-krw9g for pod openshift-network-diagnostics/network-check-target-xgn2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:10.243531 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.243458    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g podName:8a327e33-4bc2-47ae-b268-c8225e73f304 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:11.243437475 +0000 UTC m=+4.183471309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-krw9g" (UniqueName: "kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g") pod "network-check-target-xgn2n" (UID: "8a327e33-4bc2-47ae-b268-c8225e73f304") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:10.556932 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.556846    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05dde958_85c8_4751_87d7_cd7b3d9314cc.slice/crio-d45abbfdf27e0cbec5b79eb49f73302dda50b5e6956c0ba99f3d563658fea05e WatchSource:0}: Error finding container d45abbfdf27e0cbec5b79eb49f73302dda50b5e6956c0ba99f3d563658fea05e: Status 404 returned error can't find the container with id d45abbfdf27e0cbec5b79eb49f73302dda50b5e6956c0ba99f3d563658fea05e
Sep 10 19:20:10.689418 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:10.689383    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:10.689587 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:10.689524    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:10.691466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:10.691415    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:15:09 +0000 UTC" deadline="2028-03-18 04:29:27.542896267 +0000 UTC"
Sep 10 19:20:10.691466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:10.691466    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13305h9m16.851435308s"
Sep 10 19:20:10.699080 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:10.699030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bhgv7" event={"ID":"05dde958-85c8-4751-87d7-cd7b3d9314cc","Type":"ContainerStarted","Data":"d45abbfdf27e0cbec5b79eb49f73302dda50b5e6956c0ba99f3d563658fea05e"}
Sep 10 19:20:10.959615 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.959581    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3d75eb_156b_48b4_b39c_68e9907810a5.slice/crio-5aa68ef6550e926bc122b3791f8cb0180a5347fedf6042039a27d2465628c826 WatchSource:0}: Error finding container 5aa68ef6550e926bc122b3791f8cb0180a5347fedf6042039a27d2465628c826: Status 404 returned error can't find the container with id 5aa68ef6550e926bc122b3791f8cb0180a5347fedf6042039a27d2465628c826
Sep 10 19:20:10.964658 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.964584    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aeec9d3_ec71_422b_83de_777f2a89e902.slice/crio-bdd1bc83be3b9d0862cac74949066ac90089d696baf6034263264eb10ed8ff6d WatchSource:0}: Error finding container bdd1bc83be3b9d0862cac74949066ac90089d696baf6034263264eb10ed8ff6d: Status 404 returned error can't find the container with id bdd1bc83be3b9d0862cac74949066ac90089d696baf6034263264eb10ed8ff6d
Sep 10 19:20:10.966831 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.966594    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5147a1e1_114c_4f51_b475_3a681c741cb4.slice/crio-301a898df6469b39502953627393dde290e133da0a46e3f6e6fc093835bc1166 WatchSource:0}: Error finding container 301a898df6469b39502953627393dde290e133da0a46e3f6e6fc093835bc1166: Status 404 returned error can't find the container with id 301a898df6469b39502953627393dde290e133da0a46e3f6e6fc093835bc1166
Sep 10 19:20:10.967391 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.967363    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491040f7_5239_4e74_ac39_1b5a8190e333.slice/crio-5b87e7d3396118f9e1ea0d5c968ac00895d12488f81987a05b32c2f4c1f755b6 WatchSource:0}: Error finding container 5b87e7d3396118f9e1ea0d5c968ac00895d12488f81987a05b32c2f4c1f755b6: Status 404 returned error can't find the container with id 5b87e7d3396118f9e1ea0d5c968ac00895d12488f81987a05b32c2f4c1f755b6
Sep 10 19:20:10.968818 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.968744    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f2c3f98_09c4_4ecc_b015_21869f5ddb4b.slice/crio-846e4bae7cc53aaa2f4956dce35b8cd26b3602b96055d44e37384c30258b1724 WatchSource:0}: Error finding container 846e4bae7cc53aaa2f4956dce35b8cd26b3602b96055d44e37384c30258b1724: Status 404 returned error can't find the container with id 846e4bae7cc53aaa2f4956dce35b8cd26b3602b96055d44e37384c30258b1724
Sep 10 19:20:10.970258 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.970153    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6165147e_6bbf_4e85_b4f3_28b309b0004a.slice/crio-efcfd6e780a1c169f46012399e2468e718f0aa746ac9a9f18d9b02b36f63a064 WatchSource:0}: Error finding container efcfd6e780a1c169f46012399e2468e718f0aa746ac9a9f18d9b02b36f63a064: Status 404 returned error can't find the container with id efcfd6e780a1c169f46012399e2468e718f0aa746ac9a9f18d9b02b36f63a064
Sep 10 19:20:10.971588 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.971564    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c71f7e5_908d_4f2d_b087_2a5adb55ae5b.slice/crio-5f1491a00f01b74f91e98652d9bce3333f44fe1bbf04b7280f656a45f723d804 WatchSource:0}: Error finding container 5f1491a00f01b74f91e98652d9bce3333f44fe1bbf04b7280f656a45f723d804: Status 404 returned error can't find the container with id 5f1491a00f01b74f91e98652d9bce3333f44fe1bbf04b7280f656a45f723d804
Sep 10 19:20:10.972532 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:10.972504    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eca30d4_13fd_43ee_ae7d_bb61d2f0b1b2.slice/crio-2fafa4c9d0c2d7ed305a34c849f3fb969bdcbcfddeaa6d27f65a6ea67a32d040 WatchSource:0}: Error finding container 2fafa4c9d0c2d7ed305a34c849f3fb969bdcbcfddeaa6d27f65a6ea67a32d040: Status 404 returned error can't find the container with id 2fafa4c9d0c2d7ed305a34c849f3fb969bdcbcfddeaa6d27f65a6ea67a32d040
Sep 10 19:20:11.151416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.151377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:11.151612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.151430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:11.151612 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.151541    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:11.151612 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.151611    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs podName:f3338777-3059-4b20-b06a-134279c5ad91 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:13.151592413 +0000 UTC m=+6.091626229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs") pod "network-metrics-daemon-slrvh" (UID: "f3338777-3059-4b20-b06a-134279c5ad91") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:11.151794 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.151541    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:11.151794 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.151703    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret podName:e50d4c91-53ee-4496-9854-3a3abb0a1ca7 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:13.15168054 +0000 UTC m=+6.091714352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret") pod "global-pull-secret-syncer-nhl6m" (UID: "e50d4c91-53ee-4496-9854-3a3abb0a1ca7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:11.251820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.251795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:11.252023 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.252005    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:20:11.252068 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.252029    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:20:11.252068 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.252039    2562 projected.go:196] Error preparing data for projected volume kube-api-access-krw9g for pod openshift-network-diagnostics/network-check-target-xgn2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:11.252152 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.252087    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g podName:8a327e33-4bc2-47ae-b268-c8225e73f304 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:13.252072748 +0000 UTC m=+6.192106561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-krw9g" (UniqueName: "kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g") pod "network-check-target-xgn2n" (UID: "8a327e33-4bc2-47ae-b268-c8225e73f304") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:11.689566 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.689245    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:11.689566 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.689386    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:11.708984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.704116    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:11.708984 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:11.704318    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:11.715559 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.715519    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerStarted","Data":"5aa68ef6550e926bc122b3791f8cb0180a5347fedf6042039a27d2465628c826"}
Sep 10 19:20:11.721735 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.721701    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal" event={"ID":"50f4e7fca0ff25f9b39860d2874f4285","Type":"ContainerStarted","Data":"3425cda64666ed45fba75462b5e5cf3dc91d8492db87656abd6d223a3ee9624b"}
Sep 10 19:20:11.726184 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.726096    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7" event={"ID":"6165147e-6bbf-4e85-b4f3-28b309b0004a","Type":"ContainerStarted","Data":"efcfd6e780a1c169f46012399e2468e718f0aa746ac9a9f18d9b02b36f63a064"}
Sep 10 19:20:11.731262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.731190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2htct" event={"ID":"491040f7-5239-4e74-ac39-1b5a8190e333","Type":"ContainerStarted","Data":"5b87e7d3396118f9e1ea0d5c968ac00895d12488f81987a05b32c2f4c1f755b6"}
Sep 10 19:20:11.734441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.734358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xfq2p" event={"ID":"5147a1e1-114c-4f51-b475-3a681c741cb4","Type":"ContainerStarted","Data":"301a898df6469b39502953627393dde290e133da0a46e3f6e6fc093835bc1166"}
Sep 10 19:20:11.740932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.740253    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd066b74f4ff1cd5196e5b51bb3dd9d4" containerID="726f05d66e34593abe062d13a975ac45bcd7b3c78f9a28197a3f03c12862737d" exitCode=0
Sep 10 19:20:11.740932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.740351    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal" event={"ID":"dd066b74f4ff1cd5196e5b51bb3dd9d4","Type":"ContainerDied","Data":"726f05d66e34593abe062d13a975ac45bcd7b3c78f9a28197a3f03c12862737d"}
Sep 10 19:20:11.743820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.743325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"2fafa4c9d0c2d7ed305a34c849f3fb969bdcbcfddeaa6d27f65a6ea67a32d040"}
Sep 10 19:20:11.747304 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.747052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7rxpl" event={"ID":"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b","Type":"ContainerStarted","Data":"5f1491a00f01b74f91e98652d9bce3333f44fe1bbf04b7280f656a45f723d804"}
Sep 10 19:20:11.751692 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.751564    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6pr6j" event={"ID":"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b","Type":"ContainerStarted","Data":"846e4bae7cc53aaa2f4956dce35b8cd26b3602b96055d44e37384c30258b1724"}
Sep 10 19:20:11.754089 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.754058    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kq4fw" event={"ID":"5aeec9d3-ec71-422b-83de-777f2a89e902","Type":"ContainerStarted","Data":"bdd1bc83be3b9d0862cac74949066ac90089d696baf6034263264eb10ed8ff6d"}
Sep 10 19:20:11.773612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:11.773548    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-35.ec2.internal" podStartSLOduration=2.773521512 podStartE2EDuration="2.773521512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:11.747020791 +0000 UTC m=+4.687054628" watchObservedRunningTime="2026-09-10 19:20:11.773521512 +0000 UTC m=+4.713555352"
Sep 10 19:20:12.689458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:12.689366    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:12.689646 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:12.689525    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:12.775840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:12.775606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal" event={"ID":"dd066b74f4ff1cd5196e5b51bb3dd9d4","Type":"ContainerStarted","Data":"1dd85dc7d2c3d566cde09c7b709cb770a7bc96cf0ff640b49b07ea1cf09ba14c"}
Sep 10 19:20:13.171858 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:13.171765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:13.172038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:13.171863    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:13.172038 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.171975    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:13.172169 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.172042    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:13.172169 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.172060    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret podName:e50d4c91-53ee-4496-9854-3a3abb0a1ca7 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:17.172037888 +0000 UTC m=+10.112071728 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret") pod "global-pull-secret-syncer-nhl6m" (UID: "e50d4c91-53ee-4496-9854-3a3abb0a1ca7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:13.172169 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.172100    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs podName:f3338777-3059-4b20-b06a-134279c5ad91 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:17.172083215 +0000 UTC m=+10.112117034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs") pod "network-metrics-daemon-slrvh" (UID: "f3338777-3059-4b20-b06a-134279c5ad91") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:13.272608 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:13.272562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:13.272852 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.272809    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:20:13.272852 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.272834    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:20:13.272852 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.272847    2562 projected.go:196] Error preparing data for projected volume kube-api-access-krw9g for pod openshift-network-diagnostics/network-check-target-xgn2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:13.273061 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.272914    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g podName:8a327e33-4bc2-47ae-b268-c8225e73f304 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:17.272894535 +0000 UTC m=+10.212928353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-krw9g" (UniqueName: "kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g") pod "network-check-target-xgn2n" (UID: "8a327e33-4bc2-47ae-b268-c8225e73f304") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:13.695911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:13.695873    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:13.696393 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.696030    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:13.696393 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:13.696064    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:13.696393 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:13.696188    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:14.689276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:14.689197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:14.689433 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:14.689334    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:15.689915 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:15.689483    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:15.689915 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:15.689627    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:15.689915 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:15.689669    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:15.689915 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:15.689743    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:16.689834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:16.689798    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:16.690046 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:16.689957    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:17.202004 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:17.201964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:17.202175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:17.202025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:17.202175 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.202163    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:17.202290 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.202229    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs podName:f3338777-3059-4b20-b06a-134279c5ad91 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:25.202210042 +0000 UTC m=+18.142243858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs") pod "network-metrics-daemon-slrvh" (UID: "f3338777-3059-4b20-b06a-134279c5ad91") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:17.202704 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.202626    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:17.202704 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.202676    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret podName:e50d4c91-53ee-4496-9854-3a3abb0a1ca7 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:25.202660048 +0000 UTC m=+18.142693866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret") pod "global-pull-secret-syncer-nhl6m" (UID: "e50d4c91-53ee-4496-9854-3a3abb0a1ca7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:17.302797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:17.302752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:17.303028 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.303001    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:20:17.303028 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.303030    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:20:17.303190 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.303044    2562 projected.go:196] Error preparing data for projected volume kube-api-access-krw9g for pod openshift-network-diagnostics/network-check-target-xgn2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:17.303190 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.303125    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g podName:8a327e33-4bc2-47ae-b268-c8225e73f304 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:25.303103828 +0000 UTC m=+18.243137657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-krw9g" (UniqueName: "kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g") pod "network-check-target-xgn2n" (UID: "8a327e33-4bc2-47ae-b268-c8225e73f304") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:17.689907 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:17.689866    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:17.690055 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:17.689989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:17.690466 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.690122    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:17.690466 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:17.689986    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:18.689870 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:18.689836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:18.690059 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:18.689997    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:19.694030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:19.693408    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:19.694030 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:19.693524    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:19.694030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:19.693883    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:19.694030 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:19.693990    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:20.689959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:20.689698    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:20.690118 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:20.690067    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:21.693213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:21.693143    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:21.693612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:21.693144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:21.693612 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:21.693236    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:21.693752 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:21.693709    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:22.689414 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:22.689382    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:22.689577 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:22.689513    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:23.689280 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:23.689245    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:23.689715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:23.689245    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:23.689715 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:23.689392    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:23.689715 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:23.689555    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:24.689214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:24.689185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:24.689417 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:24.689288    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:25.255643 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:25.255612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:25.255816 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:25.255678    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:25.255816 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.255767    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:25.255816 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.255770    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:25.255990 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.255822    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret podName:e50d4c91-53ee-4496-9854-3a3abb0a1ca7 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:41.255804909 +0000 UTC m=+34.195838723 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret") pod "global-pull-secret-syncer-nhl6m" (UID: "e50d4c91-53ee-4496-9854-3a3abb0a1ca7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:25.255990 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.255841    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs podName:f3338777-3059-4b20-b06a-134279c5ad91 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:41.255831118 +0000 UTC m=+34.195864931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs") pod "network-metrics-daemon-slrvh" (UID: "f3338777-3059-4b20-b06a-134279c5ad91") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:25.356648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:25.356618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:25.356818 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.356787    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:20:25.356818 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.356808    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:20:25.356818 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.356817    2562 projected.go:196] Error preparing data for projected volume kube-api-access-krw9g for pod openshift-network-diagnostics/network-check-target-xgn2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:25.356992 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.356870    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g podName:8a327e33-4bc2-47ae-b268-c8225e73f304 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:41.356851168 +0000 UTC m=+34.296884982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-krw9g" (UniqueName: "kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g") pod "network-check-target-xgn2n" (UID: "8a327e33-4bc2-47ae-b268-c8225e73f304") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:25.689652 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:25.689621    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:25.690097 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.689753    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:25.690097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:25.689812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:25.690097 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:25.689941    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:26.689811 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:26.689778    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:26.690295 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:26.689922    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:27.690247 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:27.690217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:27.690750 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:27.690248    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:27.690750 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:27.690312    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:27.690750 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:27.690349    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:28.689813 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:28.689776    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:28.689984 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:28.689903    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:29.690102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.689850    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:29.690813 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.689942    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:29.690813 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:29.690131    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:29.690813 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:29.690250    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:29.824746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.824700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7rxpl" event={"ID":"3c71f7e5-908d-4f2d-b087-2a5adb55ae5b","Type":"ContainerStarted","Data":"6c235221e63228423a2f43850c30229b639ba1779591d505f0ed39462b7bf454"}
Sep 10 19:20:29.826372 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.826332    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kq4fw" event={"ID":"5aeec9d3-ec71-422b-83de-777f2a89e902","Type":"ContainerStarted","Data":"c9229524a8a6e054986c9eb059ff5dcf004d342ff2645b318dc5fa61c505da1a"}
Sep 10 19:20:29.827874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.827850    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b3d75eb-156b-48b4-b39c-68e9907810a5" containerID="d48bf7a5be0de4d1e1eaec9f1db2d51cf112da895691e6813984ba2dc254a7bf" exitCode=0
Sep 10 19:20:29.828003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.827929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerDied","Data":"d48bf7a5be0de4d1e1eaec9f1db2d51cf112da895691e6813984ba2dc254a7bf"}
Sep 10 19:20:29.829632 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.829608    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7" event={"ID":"6165147e-6bbf-4e85-b4f3-28b309b0004a","Type":"ContainerStarted","Data":"e03292b6fb0f913cdc212fd894ebce947ffb15c664c889b3e8460dbe2f704071"}
Sep 10 19:20:29.831001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.830975    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2htct" event={"ID":"491040f7-5239-4e74-ac39-1b5a8190e333","Type":"ContainerStarted","Data":"73c420bbba10b2d418f32c312ffeb849ed7f3d1b36d2839c9b93cdda0dea12aa"}
Sep 10 19:20:29.832413 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.832388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xfq2p" event={"ID":"5147a1e1-114c-4f51-b475-3a681c741cb4","Type":"ContainerStarted","Data":"50f82e5e479b60109a69bf52fbfa2fab046a3ab9170e71134af5f4a558559555"}
Sep 10 19:20:29.833870 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.833847    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bhgv7" event={"ID":"05dde958-85c8-4751-87d7-cd7b3d9314cc","Type":"ContainerStarted","Data":"4edff5ed5307a891ad346447455a9898338f569ff05c5dc4d1c9ae45e40b77e3"}
Sep 10 19:20:29.836623 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.836602    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:20:29.836943 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.836923    2562 generic.go:356] "Generic (PLEG): container finished" podID="3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2" containerID="56fb970c7866dbe7fc8123c4de397df62187498689952d008b994301d125dacb" exitCode=1
Sep 10 19:20:29.837000 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.836956    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"a5a183db58ceeea7e42ac023606a53015497629852e7a91d041368e2b3798c09"}
Sep 10 19:20:29.837038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.837012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"d82b14cc6bd9943e0c5cc442ebdbb7994f226bb5d620e56f38be8c7560284c13"}
Sep 10 19:20:29.837076 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.837052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"3a96ec1d7a4d7ee06ba219002305d6b3e4607533747adf412912ad40474d5753"}
Sep 10 19:20:29.837112 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.837090    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"717e7c29d2846a04a4bbfa858dbed11034c7376e96c683cbd5336b7c1cf52102"}
Sep 10 19:20:29.837146 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.837117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerDied","Data":"56fb970c7866dbe7fc8123c4de397df62187498689952d008b994301d125dacb"}
Sep 10 19:20:29.837178 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.837146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"9ea8e80670b9e3b727a1623277a7c9487a76cec019236a06cf0ca306a6442dc3"}
Sep 10 19:20:29.842406 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.842352    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7rxpl" podStartSLOduration=4.155718374 podStartE2EDuration="21.84233511s" totalImagesPullingTime="17.686616736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:29.842089333 +0000 UTC m=+22.782123166" watchObservedRunningTime="2026-09-10 19:20:29.84233511 +0000 UTC m=+22.782368944"
Sep 10 19:20:29.842758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.842731    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-35.ec2.internal" podStartSLOduration=20.842719037 podStartE2EDuration="20.842719037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:12.804170842 +0000 UTC m=+5.744204680" watchObservedRunningTime="2026-09-10 19:20:29.842719037 +0000 UTC m=+22.782752871"
Sep 10 19:20:29.859026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.858969    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2htct" podStartSLOduration=4.130171769 podStartE2EDuration="21.858953093s" totalImagesPullingTime="17.728781324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:29.858441881 +0000 UTC m=+22.798475713" watchObservedRunningTime="2026-09-10 19:20:29.858953093 +0000 UTC m=+22.798986925"
Sep 10 19:20:29.875927 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.875854    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xfq2p" podStartSLOduration=4.146745855 podStartE2EDuration="21.875839035s" totalImagesPullingTime="17.72909318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:29.875534404 +0000 UTC m=+22.815568238" watchObservedRunningTime="2026-09-10 19:20:29.875839035 +0000 UTC m=+22.815872867"
Sep 10 19:20:29.889945 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.889873    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bhgv7" podStartSLOduration=4.058430688 podStartE2EDuration="21.889858643s" totalImagesPullingTime="17.831427955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:29.889827054 +0000 UTC m=+22.829860887" watchObservedRunningTime="2026-09-10 19:20:29.889858643 +0000 UTC m=+22.829892470"
Sep 10 19:20:29.948115 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:29.948062    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kq4fw" podStartSLOduration=3.754602303 podStartE2EDuration="21.948046902s" totalImagesPullingTime="18.193444599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:29.947766788 +0000 UTC m=+22.887800622" watchObservedRunningTime="2026-09-10 19:20:29.948046902 +0000 UTC m=+22.888080734"
Sep 10 19:20:30.689207 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:30.689175    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:30.689371 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:30.689306    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:30.788949 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:30.788656    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 19:20:30.841523 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:30.841482    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6pr6j" event={"ID":"6f2c3f98-09c4-4ecc-b015-21869f5ddb4b","Type":"ContainerStarted","Data":"30d2df81b45b458ae08823b4acf2b61264fe8be0e70bd7b9d63f50eb6c6339f7"}
Sep 10 19:20:30.844130 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:30.844094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7" event={"ID":"6165147e-6bbf-4e85-b4f3-28b309b0004a","Type":"ContainerStarted","Data":"c3eba1338398a0e575a053c04bfe73bef2742071ef784dbfd16ced7939f4d447"}
Sep 10 19:20:30.856762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:30.856698    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6pr6j" podStartSLOduration=5.166965994 podStartE2EDuration="22.856673876s" totalImagesPullingTime="17.689707882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:30.856449025 +0000 UTC m=+23.796482874" watchObservedRunningTime="2026-09-10 19:20:30.856673876 +0000 UTC m=+23.796707713"
Sep 10 19:20:31.505858 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.505779    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:31.506426 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.506406    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:31.690119 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.690088    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:31.690119 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.690106    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:31.690363 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:31.690241    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:31.690363 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:31.690334    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:31.696391 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.696298    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T19:20:30.788827121Z","UUID":"f1c9c67b-4ee1-4c3f-9361-81e3c8fa9830","Handler":null,"Name":"","Endpoint":""}
Sep 10 19:20:31.697998 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.697966    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 19:20:31.698144 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.698007    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 19:20:31.849643 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.849578    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:20:31.850012 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.849959    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"b4350c772d0f39adbd85fb033f3d55c41e49fefc1b0e0b35d035a0842574c109"}
Sep 10 19:20:31.850223 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.850198    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:31.850673 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:31.850651    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7rxpl"
Sep 10 19:20:32.689487 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:32.689451    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:32.689667 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:32.689597    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:32.854863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:32.854829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7" event={"ID":"6165147e-6bbf-4e85-b4f3-28b309b0004a","Type":"ContainerStarted","Data":"2c7230661513ac0cce552c89cc03dda30c821b71875082d02ef464db906885bc"}
Sep 10 19:20:32.878490 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:32.878432    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g4b7" podStartSLOduration=4.337933336 podStartE2EDuration="24.878412074s" totalImagesPullingTime="20.540478738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:32.877693671 +0000 UTC m=+25.817727507" watchObservedRunningTime="2026-09-10 19:20:32.878412074 +0000 UTC m=+25.818445900"
Sep 10 19:20:33.689601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:33.689576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:33.689859 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:33.689576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:33.689859 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:33.689709    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:33.689859 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:33.689749    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:33.862751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:33.862673    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:20:33.863288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:33.863242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"db8e2a097271602b9c76c2a9d90c49ec1479a8b6188c2208302e3bb5bf054d23"}
Sep 10 19:20:33.863832 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:33.863803    2562 scope.go:117] "RemoveContainer" containerID="56fb970c7866dbe7fc8123c4de397df62187498689952d008b994301d125dacb"
Sep 10 19:20:34.689160 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.689120    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:34.689400 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:34.689240    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:34.881551 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.881521    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:20:34.883588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.882376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" event={"ID":"3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2","Type":"ContainerStarted","Data":"8da3cead6eac59a9a6ff4a9c41e22002fd71d851ebe6151c6f7deb40b553bd76"}
Sep 10 19:20:34.885466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.883969    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:34.885466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.884112    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:34.885466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.884204    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:34.909610 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.909300    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:34.909610 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.909511    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:20:34.917274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:34.917213    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n" podStartSLOduration=9.175359802 podStartE2EDuration="26.917191157s" totalImagesPullingTime="17.741831355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:34.914805362 +0000 UTC m=+27.854839196" watchObservedRunningTime="2026-09-10 19:20:34.917191157 +0000 UTC m=+27.857224993"
Sep 10 19:20:35.693321 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.693294    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:35.693505 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.693294    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:35.693505 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:35.693399    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:35.693624 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:35.693520    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:35.810289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.810256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xgn2n"]
Sep 10 19:20:35.814430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.814371    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nhl6m"]
Sep 10 19:20:35.815167 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.815143    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-slrvh"]
Sep 10 19:20:35.815298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.815275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:35.815426 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:35.815394    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:35.884777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.884746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:35.884777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:35.884763    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:35.885291 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:35.884849    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:35.885291 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:35.885240    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:36.889193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:36.889164    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b3d75eb-156b-48b4-b39c-68e9907810a5" containerID="c1582cd770ad7d90b1474da93e2721088102a0684cebf32e660df9d4820d5ddb" exitCode=0
Sep 10 19:20:36.889776 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:36.889250    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerDied","Data":"c1582cd770ad7d90b1474da93e2721088102a0684cebf32e660df9d4820d5ddb"}
Sep 10 19:20:37.693871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:37.693412    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:37.693871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:37.693429    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:37.693871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:37.693422    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:37.693871 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:37.693534    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:37.693871 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:37.693657    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:37.693871 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:37.693745    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:38.899133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:38.898935    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b3d75eb-156b-48b4-b39c-68e9907810a5" containerID="831a36e284fcc0accfc738fa0d3add3335e7c784682fd95a904883c6f847ec81" exitCode=0
Sep 10 19:20:38.899133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:38.899010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerDied","Data":"831a36e284fcc0accfc738fa0d3add3335e7c784682fd95a904883c6f847ec81"}
Sep 10 19:20:39.689744 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:39.689706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:39.689932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:39.689706    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:39.689932 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:39.689830    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:39.689932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:39.689711    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:39.689932 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:39.689921    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:39.690077 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:39.690011    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:40.905754 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:40.905720    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b3d75eb-156b-48b4-b39c-68e9907810a5" containerID="7f4f99b98fbe7463308e8ff397527c42f0928a0ba8e5d772fe3c9fffa9967205" exitCode=0
Sep 10 19:20:40.906159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:40.905767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerDied","Data":"7f4f99b98fbe7463308e8ff397527c42f0928a0ba8e5d772fe3c9fffa9967205"}
Sep 10 19:20:41.269116 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.269004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:41.269116 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.269109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:41.269355 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.269162    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:41.269355 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.269185    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:41.269355 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.269232    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs podName:f3338777-3059-4b20-b06a-134279c5ad91 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.26921811 +0000 UTC m=+66.209251926 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs") pod "network-metrics-daemon-slrvh" (UID: "f3338777-3059-4b20-b06a-134279c5ad91") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:20:41.269355 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.269247    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret podName:e50d4c91-53ee-4496-9854-3a3abb0a1ca7 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.269241154 +0000 UTC m=+66.209274970 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret") pod "global-pull-secret-syncer-nhl6m" (UID: "e50d4c91-53ee-4496-9854-3a3abb0a1ca7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:20:41.370199 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.370165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:41.370375 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.370310    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:20:41.370375 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.370329    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:20:41.370375 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.370340    2562 projected.go:196] Error preparing data for projected volume kube-api-access-krw9g for pod openshift-network-diagnostics/network-check-target-xgn2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:41.370547 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.370406    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g podName:8a327e33-4bc2-47ae-b268-c8225e73f304 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.37038723 +0000 UTC m=+66.310421065 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-krw9g" (UniqueName: "kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g") pod "network-check-target-xgn2n" (UID: "8a327e33-4bc2-47ae-b268-c8225e73f304") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:20:41.689584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.689545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:41.689736 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.689545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:41.689736 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.689672    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhl6m" podUID="e50d4c91-53ee-4496-9854-3a3abb0a1ca7"
Sep 10 19:20:41.689819 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.689760    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xgn2n" podUID="8a327e33-4bc2-47ae-b268-c8225e73f304"
Sep 10 19:20:41.689819 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.689551    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:41.689921 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:41.689848    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-slrvh" podUID="f3338777-3059-4b20-b06a-134279c5ad91"
Sep 10 19:20:41.850669 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.850586    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-35.ec2.internal" event="NodeReady"
Sep 10 19:20:41.850797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.850714    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 19:20:41.884042 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.884004    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78875ff86d-blbk5"]
Sep 10 19:20:41.887402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.887376    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.890711 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.890678    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 19:20:41.891018 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.890944    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 19:20:41.892032 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.891764    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-s4d5w"]
Sep 10 19:20:41.892187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.892170    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 19:20:41.892315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.892290    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x5vxr\""
Sep 10 19:20:41.895976 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.895768    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"]
Sep 10 19:20:41.895976 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.895923    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:41.896775 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.896756    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 19:20:41.898453 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.898434    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 19:20:41.898538 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.898439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.898930 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.898915    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.899218 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.899201    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 19:20:41.899409 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.899393    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9dc57874b-zzp9h"]
Sep 10 19:20:41.900605 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.899760    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:41.900605 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.900145    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-c5cfl\""
Sep 10 19:20:41.904848 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.904823    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.905099 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.905079    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 19:20:41.905199 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.905122    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kfzpl\""
Sep 10 19:20:41.906238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.906026    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 19:20:41.906238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.906175    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.907209 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.907188    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"]
Sep 10 19:20:41.908014 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.907298    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:41.908404 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.908388    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 19:20:41.910872 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.910851    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"]
Sep 10 19:20:41.911541 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.911407    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 19:20:41.911541 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.911420    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qfppk\""
Sep 10 19:20:41.911541 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.911414    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 19:20:41.912204 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.911754    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 19:20:41.912204 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.911802    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 19:20:41.912204 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.911854    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.912204 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.911947    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.912204 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.912116    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:41.914497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.914476    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78875ff86d-blbk5"]
Sep 10 19:20:41.914603 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.914515    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-s4d5w"]
Sep 10 19:20:41.914603 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.914553    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:41.915266 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.915241    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.915384 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.915362    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.915562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.915547    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 19:20:41.915611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.915566    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 19:20:41.915789 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.915772    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-49mhm\""
Sep 10 19:20:41.918318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.918298    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"]
Sep 10 19:20:41.921760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.921742    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"]
Sep 10 19:20:41.922775 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.922741    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"]
Sep 10 19:20:41.924105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.923734    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.924105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.923684    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-t7892\""
Sep 10 19:20:41.924105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.923676    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 19:20:41.924105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.924067    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.924105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.924073    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 19:20:41.927275 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.927256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9dc57874b-zzp9h"]
Sep 10 19:20:41.927374 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.927292    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-wqgz2"]
Sep 10 19:20:41.927436 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.927386    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"
Sep 10 19:20:41.930819 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.930803    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:41.931326 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.930827    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rzn7d"]
Sep 10 19:20:41.931630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.931614    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-w2x4w\""
Sep 10 19:20:41.932060 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.932042    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.932812 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.932790    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.934959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.934939    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rkxv6"]
Sep 10 19:20:41.935373 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.935356    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d"
Sep 10 19:20:41.937565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.937547    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 19:20:41.938089 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.937987    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 19:20:41.940303 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.940134    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cqwnz\""
Sep 10 19:20:41.940303 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.940215    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.943172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.940963    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.943172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.941622    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dp8ct\""
Sep 10 19:20:41.944603 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.944422    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-7b5pk"]
Sep 10 19:20:41.948023 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.945354    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:41.948499 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.948420    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.950248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.949656    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gl8x4\""
Sep 10 19:20:41.950248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.949903    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 19:20:41.950248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.950046    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.950725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.950696    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nh2g2"]
Sep 10 19:20:41.951604 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.950939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:41.956337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.955836    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"]
Sep 10 19:20:41.956337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.956287    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2k4md\""
Sep 10 19:20:41.956457 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.956448    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.956508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.956460    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 19:20:41.956974 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.956782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:41.958102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.957387    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.958102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.957814    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 19:20:41.959945 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.959923    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 19:20:41.960185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960166    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 19:20:41.960369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960353    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2kts8\""
Sep 10 19:20:41.960806 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960758    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"]
Sep 10 19:20:41.960806 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960801    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"]
Sep 10 19:20:41.961147 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960823    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-wqgz2"]
Sep 10 19:20:41.961147 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960844    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rzn7d"]
Sep 10 19:20:41.961147 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960907    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nh2g2"]
Sep 10 19:20:41.961147 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960931    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rkxv6"]
Sep 10 19:20:41.961147 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:41.961147 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.960949    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"]
Sep 10 19:20:41.963213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.963191    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 19:20:41.963846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.963828    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 19:20:41.964031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.964012    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ss72r\""
Sep 10 19:20:41.964136 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.963839    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 19:20:41.964204 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.963874    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:20:41.965906 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.965861    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-7b5pk"]
Sep 10 19:20:41.974687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-trusted-ca\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.974805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-stats-auth\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:41.974805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974740    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-bound-sa-token\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.974805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974769    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr9jl\" (UniqueName: \"kubernetes.io/projected/33fdd110-50e5-49e9-93f7-c57f7cfa5764-kube-api-access-vr9jl\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:41.974973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974830    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:41.974973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974880    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20d869f4-b859-45f8-a21d-d90bf157087f-serving-cert\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:41.974973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph7b9\" (UniqueName: \"kubernetes.io/projected/1a7b5479-66fb-4a0a-83d6-1a50656b07af-kube-api-access-ph7b9\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:41.975125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.974975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d869f4-b859-45f8-a21d-d90bf157087f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:41.975125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-image-registry-private-configuration\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.975125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a7b5479-66fb-4a0a-83d6-1a50656b07af-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:41.975267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20d869f4-b859-45f8-a21d-d90bf157087f-tmp\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:41.975267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975169    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d869f4-b859-45f8-a21d-d90bf157087f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:41.975267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-ca-trust-extracted\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.975267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975231    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:41.975565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-installation-pull-secrets\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.975565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:41.975565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975344    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/33fdd110-50e5-49e9-93f7-c57f7cfa5764-config\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:41.975565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.975769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlchs\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-kube-api-access-jlchs\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.975769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33fdd110-50e5-49e9-93f7-c57f7cfa5764-serving-cert\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:41.975769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:41.975769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975737    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk2p8\" (UniqueName: \"kubernetes.io/projected/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-kube-api-access-jk2p8\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:41.975769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/20d869f4-b859-45f8-a21d-d90bf157087f-snapshots\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:41.976026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9fzx\" (UniqueName: \"kubernetes.io/projected/20d869f4-b859-45f8-a21d-d90bf157087f-kube-api-access-r9fzx\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:41.976026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975824    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-certificates\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:41.976026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975866    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a7b5479-66fb-4a0a-83d6-1a50656b07af-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:41.976026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-default-certificate\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:41.976026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:41.975979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c87n\" (UniqueName: \"kubernetes.io/projected/fc444d45-54f7-4c61-a08d-34658072f79a-kube-api-access-4c87n\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.076686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.076651    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ls6k\" (UniqueName: \"kubernetes.io/projected/ac4033cf-e6b0-4176-9105-4d945dc166f1-kube-api-access-6ls6k\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:42.076916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.076708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.076916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.076736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/20d869f4-b859-45f8-a21d-d90bf157087f-snapshots\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.076916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.076761    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-certificates\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.076916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.076781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:42.076916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.076822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-stats-auth\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.076916 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.076844    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:20:42.076916 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.076864    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78875ff86d-blbk5: secret "image-registry-tls" not found
Sep 10 19:20:42.077187 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.076947    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls podName:e7f4b902-9333-4b0e-bd8f-3c01620d0e79 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:42.576927024 +0000 UTC m=+35.516960856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls") pod "image-registry-78875ff86d-blbk5" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79") : secret "image-registry-tls" not found
Sep 10 19:20:42.077187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.076848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr9jl\" (UniqueName: \"kubernetes.io/projected/33fdd110-50e5-49e9-93f7-c57f7cfa5764-kube-api-access-vr9jl\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:42.077187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077091    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-serving-cert\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.077187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.077187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.077187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph7b9\" (UniqueName: \"kubernetes.io/projected/1a7b5479-66fb-4a0a-83d6-1a50656b07af-kube-api-access-ph7b9\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:42.077371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrqpt\" (UniqueName: \"kubernetes.io/projected/8fa87b08-6c57-4461-a536-f2a9adeac807-kube-api-access-hrqpt\") pod \"ingress-canary-rkxv6\" (UID: \"8fa87b08-6c57-4461-a536-f2a9adeac807\") " pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:42.077371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077226    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-image-registry-private-configuration\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.077371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077246    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa87b08-6c57-4461-a536-f2a9adeac807-cert\") pod \"ingress-canary-rkxv6\" (UID: \"8fa87b08-6c57-4461-a536-f2a9adeac807\") " pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:42.077371 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.077279    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:42.57725978 +0000 UTC m=+35.517293609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : configmap references non-existent config key: service-ca.crt
Sep 10 19:20:42.077371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077318    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d869f4-b859-45f8-a21d-d90bf157087f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.077371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.077371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077352    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-certificates\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb8x8\" (UniqueName: \"kubernetes.io/projected/6097bb6c-42d7-434d-8fd2-e02d0514e025-kube-api-access-lb8x8\") pod \"network-check-source-685456655f-rzn7d\" (UID: \"6097bb6c-42d7-434d-8fd2-e02d0514e025\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077413    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlchs\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-kube-api-access-jlchs\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33fdd110-50e5-49e9-93f7-c57f7cfa5764-serving-cert\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/20d869f4-b859-45f8-a21d-d90bf157087f-snapshots\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk2p8\" (UniqueName: \"kubernetes.io/projected/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-kube-api-access-jk2p8\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9fzx\" (UniqueName: \"kubernetes.io/projected/20d869f4-b859-45f8-a21d-d90bf157087f-kube-api-access-r9fzx\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-config\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.077579    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:20:42.077649 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.077626    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:42.577610479 +0000 UTC m=+35.517644304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : secret "router-metrics-certs-default" not found
Sep 10 19:20:42.078183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ln7d\" (UniqueName: \"kubernetes.io/projected/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-kube-api-access-5ln7d\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.078183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077866    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a7b5479-66fb-4a0a-83d6-1a50656b07af-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:42.078183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.077971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-default-certificate\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.078336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c87n\" (UniqueName: \"kubernetes.io/projected/fc444d45-54f7-4c61-a08d-34658072f79a-kube-api-access-4c87n\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.078391 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078333    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-trusted-ca\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.078391 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-bound-sa-token\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.078491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g55vh\" (UniqueName: \"kubernetes.io/projected/ccd4a7aa-770f-4dba-a34d-490ba52734b6-kube-api-access-g55vh\") pod \"volume-data-source-validator-695c895bd6-cv7hn\" (UID: \"ccd4a7aa-770f-4dba-a34d-490ba52734b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"
Sep 10 19:20:42.078491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d869f4-b859-45f8-a21d-d90bf157087f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.078588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078488    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-config-volume\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.078588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqj29\" (UniqueName: \"kubernetes.io/projected/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-kube-api-access-dqj29\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.078588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a7b5479-66fb-4a0a-83d6-1a50656b07af-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:42.078730 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-tmp-dir\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.078730 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078627    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:42.078730 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20d869f4-b859-45f8-a21d-d90bf157087f-tmp\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.078869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04f372b4-a549-4f39-b87f-7f53a360a242-nginx-conf\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:42.078869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-ca-trust-extracted\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.078869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20d869f4-b859-45f8-a21d-d90bf157087f-serving-cert\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.078869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-installation-pull-secrets\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.079070 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:42.079070 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:42.079070 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/33fdd110-50e5-49e9-93f7-c57f7cfa5764-config\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:42.079070 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.078945    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-trusted-ca\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.079278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.079253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d869f4-b859-45f8-a21d-d90bf157087f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.079332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.079289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20d869f4-b859-45f8-a21d-d90bf157087f-tmp\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.079600 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.079574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-ca-trust-extracted\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.079754 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.079538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a7b5479-66fb-4a0a-83d6-1a50656b07af-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:42.080733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.080534    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20d869f4-b859-45f8-a21d-d90bf157087f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.080939 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.080917    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:42.081032 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.080981    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls podName:e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d nodeName:}" failed. No retries permitted until 2026-09-10 19:20:42.580964605 +0000 UTC m=+35.520998422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-sk5q6" (UID: "e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:42.081131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.081108    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-trusted-ca\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.081588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.081562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/33fdd110-50e5-49e9-93f7-c57f7cfa5764-config\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:42.082696 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.082671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a7b5479-66fb-4a0a-83d6-1a50656b07af-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:42.083034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.082945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-image-registry-private-configuration\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.083034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.082964    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-default-certificate\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.083881 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.083769    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33fdd110-50e5-49e9-93f7-c57f7cfa5764-serving-cert\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:42.083987 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.083913    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20d869f4-b859-45f8-a21d-d90bf157087f-serving-cert\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.084847 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.084190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-stats-auth\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.086538 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.086508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-installation-pull-secrets\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.087387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.087362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9fzx\" (UniqueName: \"kubernetes.io/projected/20d869f4-b859-45f8-a21d-d90bf157087f-kube-api-access-r9fzx\") pod \"insights-operator-76bcd6ccf5-s4d5w\" (UID: \"20d869f4-b859-45f8-a21d-d90bf157087f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.087488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.087348    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph7b9\" (UniqueName: \"kubernetes.io/projected/1a7b5479-66fb-4a0a-83d6-1a50656b07af-kube-api-access-ph7b9\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8szwh\" (UID: \"1a7b5479-66fb-4a0a-83d6-1a50656b07af\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:42.088014 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.087994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk2p8\" (UniqueName: \"kubernetes.io/projected/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-kube-api-access-jk2p8\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:42.088638 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.088432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr9jl\" (UniqueName: \"kubernetes.io/projected/33fdd110-50e5-49e9-93f7-c57f7cfa5764-kube-api-access-vr9jl\") pod \"service-ca-operator-859f94c4b-6rf67\" (UID: \"33fdd110-50e5-49e9-93f7-c57f7cfa5764\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:42.096488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.096459    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c87n\" (UniqueName: \"kubernetes.io/projected/fc444d45-54f7-4c61-a08d-34658072f79a-kube-api-access-4c87n\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.096619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.096596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlchs\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-kube-api-access-jlchs\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.097535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.097512    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-bound-sa-token\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.179567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqj29\" (UniqueName: \"kubernetes.io/projected/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-kube-api-access-dqj29\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.179567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-tmp-dir\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.179802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:42.179802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04f372b4-a549-4f39-b87f-7f53a360a242-nginx-conf\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:42.179802 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.179718    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:20:42.179802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-trusted-ca\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.179802 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.179799    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls podName:ac4033cf-e6b0-4176-9105-4d945dc166f1 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:42.67977828 +0000 UTC m=+35.619812106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8dfpf" (UID: "ac4033cf-e6b0-4176-9105-4d945dc166f1") : secret "samples-operator-tls" not found
Sep 10 19:20:42.180093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179836    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ls6k\" (UniqueName: \"kubernetes.io/projected/ac4033cf-e6b0-4176-9105-4d945dc166f1-kube-api-access-6ls6k\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:42.180093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:42.180093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.179991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-tmp-dir\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.180093 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.180012    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:20:42.180093 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.180081    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert podName:04f372b4-a549-4f39-b87f-7f53a360a242 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:42.680063794 +0000 UTC m=+35.620097622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-wqgz2" (UID: "04f372b4-a549-4f39-b87f-7f53a360a242") : secret "networking-console-plugin-cert" not found
Sep 10 19:20:42.180332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-serving-cert\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.180332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180160    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.180332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrqpt\" (UniqueName: \"kubernetes.io/projected/8fa87b08-6c57-4461-a536-f2a9adeac807-kube-api-access-hrqpt\") pod \"ingress-canary-rkxv6\" (UID: \"8fa87b08-6c57-4461-a536-f2a9adeac807\") " pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:42.180332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa87b08-6c57-4461-a536-f2a9adeac807-cert\") pod \"ingress-canary-rkxv6\" (UID: \"8fa87b08-6c57-4461-a536-f2a9adeac807\") " pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:42.180332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb8x8\" (UniqueName: \"kubernetes.io/projected/6097bb6c-42d7-434d-8fd2-e02d0514e025-kube-api-access-lb8x8\") pod \"network-check-source-685456655f-rzn7d\" (UID: \"6097bb6c-42d7-434d-8fd2-e02d0514e025\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d"
Sep 10 19:20:42.180560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180335    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-config\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.180560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180354    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04f372b4-a549-4f39-b87f-7f53a360a242-nginx-conf\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:42.180560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ln7d\" (UniqueName: \"kubernetes.io/projected/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-kube-api-access-5ln7d\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.180560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g55vh\" (UniqueName: \"kubernetes.io/projected/ccd4a7aa-770f-4dba-a34d-490ba52734b6-kube-api-access-g55vh\") pod \"volume-data-source-validator-695c895bd6-cv7hn\" (UID: \"ccd4a7aa-770f-4dba-a34d-490ba52734b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"
Sep 10 19:20:42.180560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-config-volume\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.180796 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.180298    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:20:42.180796 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.180690    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls podName:c6f9c1d8-a78a-4541-bb95-e368edfee6fe nodeName:}" failed. No retries permitted until 2026-09-10 19:20:42.680673863 +0000 UTC m=+35.620707676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls") pod "dns-default-nh2g2" (UID: "c6f9c1d8-a78a-4541-bb95-e368edfee6fe") : secret "dns-default-metrics-tls" not found
Sep 10 19:20:42.180929 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.180830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-trusted-ca\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.181047 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.181025    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-config-volume\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.181447 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.181424    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-config\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.182913 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.182870    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-serving-cert\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.183484 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.183458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fa87b08-6c57-4461-a536-f2a9adeac807-cert\") pod \"ingress-canary-rkxv6\" (UID: \"8fa87b08-6c57-4461-a536-f2a9adeac807\") " pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:42.191000 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.190971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqj29\" (UniqueName: \"kubernetes.io/projected/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-kube-api-access-dqj29\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.191258 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.191217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrqpt\" (UniqueName: \"kubernetes.io/projected/8fa87b08-6c57-4461-a536-f2a9adeac807-kube-api-access-hrqpt\") pod \"ingress-canary-rkxv6\" (UID: \"8fa87b08-6c57-4461-a536-f2a9adeac807\") " pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:42.191552 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.191526    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ls6k\" (UniqueName: \"kubernetes.io/projected/ac4033cf-e6b0-4176-9105-4d945dc166f1-kube-api-access-6ls6k\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:42.192289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.192267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g55vh\" (UniqueName: \"kubernetes.io/projected/ccd4a7aa-770f-4dba-a34d-490ba52734b6-kube-api-access-g55vh\") pod \"volume-data-source-validator-695c895bd6-cv7hn\" (UID: \"ccd4a7aa-770f-4dba-a34d-490ba52734b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"
Sep 10 19:20:42.192983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.192961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ln7d\" (UniqueName: \"kubernetes.io/projected/2f2e6b2f-afbe-45bb-84bc-582ace2e0379-kube-api-access-5ln7d\") pod \"console-operator-85b454874b-7b5pk\" (UID: \"2f2e6b2f-afbe-45bb-84bc-582ace2e0379\") " pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.193427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.193407    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb8x8\" (UniqueName: \"kubernetes.io/projected/6097bb6c-42d7-434d-8fd2-e02d0514e025-kube-api-access-lb8x8\") pod \"network-check-source-685456655f-rzn7d\" (UID: \"6097bb6c-42d7-434d-8fd2-e02d0514e025\") " pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d"
Sep 10 19:20:42.211284 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.211249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w"
Sep 10 19:20:42.235123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.235077    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"
Sep 10 19:20:42.250067 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.249983    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"
Sep 10 19:20:42.262750 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.262674    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"
Sep 10 19:20:42.284296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.284121    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d"
Sep 10 19:20:42.287302 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.287217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rkxv6"
Sep 10 19:20:42.293953 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.293903    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:20:42.511211 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.511138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-s4d5w"]
Sep 10 19:20:42.514739 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:42.514698    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d869f4_b859_45f8_a21d_d90bf157087f.slice/crio-37d008f31dee758188a46a8734feb0bbb03fc6187625cc5842e815bf130399cf WatchSource:0}: Error finding container 37d008f31dee758188a46a8734feb0bbb03fc6187625cc5842e815bf130399cf: Status 404 returned error can't find the container with id 37d008f31dee758188a46a8734feb0bbb03fc6187625cc5842e815bf130399cf
Sep 10 19:20:42.537276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.537115    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67"]
Sep 10 19:20:42.540024 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:42.539990    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fdd110_50e5_49e9_93f7_c57f7cfa5764.slice/crio-eac8d959f7ef82cb4d5b67684ab5429e687ca4992c656dcc95d5263403bdb834 WatchSource:0}: Error finding container eac8d959f7ef82cb4d5b67684ab5429e687ca4992c656dcc95d5263403bdb834: Status 404 returned error can't find the container with id eac8d959f7ef82cb4d5b67684ab5429e687ca4992c656dcc95d5263403bdb834
Sep 10 19:20:42.558794 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.558763    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh"]
Sep 10 19:20:42.561581 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:42.561551    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7b5479_66fb_4a0a_83d6_1a50656b07af.slice/crio-ae4bc10f525ed640586cae7e3782c3a25fb0bf85f93181ad2a840ffb8df389d9 WatchSource:0}: Error finding container ae4bc10f525ed640586cae7e3782c3a25fb0bf85f93181ad2a840ffb8df389d9: Status 404 returned error can't find the container with id ae4bc10f525ed640586cae7e3782c3a25fb0bf85f93181ad2a840ffb8df389d9
Sep 10 19:20:42.569972 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.569945    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn"]
Sep 10 19:20:42.573031 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:42.573005    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd4a7aa_770f_4dba_a34d_490ba52734b6.slice/crio-44684eef8f1388e9ac635bc7ef4eab5d4cce5a0ed92081e312e5b3b0bdad1a4a WatchSource:0}: Error finding container 44684eef8f1388e9ac635bc7ef4eab5d4cce5a0ed92081e312e5b3b0bdad1a4a: Status 404 returned error can't find the container with id 44684eef8f1388e9ac635bc7ef4eab5d4cce5a0ed92081e312e5b3b0bdad1a4a
Sep 10 19:20:42.587737 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.587699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:42.587902 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.587743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:42.587902 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.587814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.587902 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.587865    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:42.587902 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.587871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:42.588116 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.587960    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:20:42.588116 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.587981    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78875ff86d-blbk5: secret "image-registry-tls" not found
Sep 10 19:20:42.588116 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.587996    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:20:42.588116 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.587971    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls podName:e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d nodeName:}" failed. No retries permitted until 2026-09-10 19:20:43.587940345 +0000 UTC m=+36.527974179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-sk5q6" (UID: "e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:42.588116 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.588052    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:43.588030233 +0000 UTC m=+36.528064044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : configmap references non-existent config key: service-ca.crt
Sep 10 19:20:42.588116 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.588084    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls podName:e7f4b902-9333-4b0e-bd8f-3c01620d0e79 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:43.588074127 +0000 UTC m=+36.528107938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls") pod "image-registry-78875ff86d-blbk5" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79") : secret "image-registry-tls" not found
Sep 10 19:20:42.588116 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.588101    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:43.588092595 +0000 UTC m=+36.528126408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : secret "router-metrics-certs-default" not found
Sep 10 19:20:42.596962 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.596938    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-rzn7d"]
Sep 10 19:20:42.599606 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:42.599577    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6097bb6c_42d7_434d_8fd2_e02d0514e025.slice/crio-7bac297994513b157c40522389de8df6d69922a94c6f2ebdfdded683f0664fe3 WatchSource:0}: Error finding container 7bac297994513b157c40522389de8df6d69922a94c6f2ebdfdded683f0664fe3: Status 404 returned error can't find the container with id 7bac297994513b157c40522389de8df6d69922a94c6f2ebdfdded683f0664fe3
Sep 10 19:20:42.626022 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.625984    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rkxv6"]
Sep 10 19:20:42.628492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.628463    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-7b5pk"]
Sep 10 19:20:42.629419 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:42.629328    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa87b08_6c57_4461_a536_f2a9adeac807.slice/crio-851accf9dc194b557e510c186986582c8da9bef11b4042550ad3f1ea1c37fb10 WatchSource:0}: Error finding container 851accf9dc194b557e510c186986582c8da9bef11b4042550ad3f1ea1c37fb10: Status 404 returned error can't find the container with id 851accf9dc194b557e510c186986582c8da9bef11b4042550ad3f1ea1c37fb10
Sep 10 19:20:42.631769 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:20:42.631700    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2e6b2f_afbe_45bb_84bc_582ace2e0379.slice/crio-1d37d04c7a2c0a49405e8d30a7c2a9d9ccd700e311ab596c1cdb1f29b50214d6 WatchSource:0}: Error finding container 1d37d04c7a2c0a49405e8d30a7c2a9d9ccd700e311ab596c1cdb1f29b50214d6: Status 404 returned error can't find the container with id 1d37d04c7a2c0a49405e8d30a7c2a9d9ccd700e311ab596c1cdb1f29b50214d6
Sep 10 19:20:42.689187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.689154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:42.689369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.689235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:42.689369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.689294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:42.689369 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.689339    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:20:42.689536 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.689394    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:20:42.689536 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.689438    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls podName:ac4033cf-e6b0-4176-9105-4d945dc166f1 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:43.689418455 +0000 UTC m=+36.629452285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8dfpf" (UID: "ac4033cf-e6b0-4176-9105-4d945dc166f1") : secret "samples-operator-tls" not found
Sep 10 19:20:42.689536 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.689443    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:20:42.689536 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.689460    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert podName:04f372b4-a549-4f39-b87f-7f53a360a242 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:43.689448667 +0000 UTC m=+36.629482478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-wqgz2" (UID: "04f372b4-a549-4f39-b87f-7f53a360a242") : secret "networking-console-plugin-cert" not found
Sep 10 19:20:42.689536 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:42.689488    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls podName:c6f9c1d8-a78a-4541-bb95-e368edfee6fe nodeName:}" failed. No retries permitted until 2026-09-10 19:20:43.689472541 +0000 UTC m=+36.629506367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls") pod "dns-default-nh2g2" (UID: "c6f9c1d8-a78a-4541-bb95-e368edfee6fe") : secret "dns-default-metrics-tls" not found
Sep 10 19:20:42.778083 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.777989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:42.885255 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.885218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:42.913579 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.913537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn" event={"ID":"ccd4a7aa-770f-4dba-a34d-490ba52734b6","Type":"ContainerStarted","Data":"44684eef8f1388e9ac635bc7ef4eab5d4cce5a0ed92081e312e5b3b0bdad1a4a"}
Sep 10 19:20:42.915022 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.914990    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh" event={"ID":"1a7b5479-66fb-4a0a-83d6-1a50656b07af","Type":"ContainerStarted","Data":"ae4bc10f525ed640586cae7e3782c3a25fb0bf85f93181ad2a840ffb8df389d9"}
Sep 10 19:20:42.916320 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.916276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w" event={"ID":"20d869f4-b859-45f8-a21d-d90bf157087f","Type":"ContainerStarted","Data":"37d008f31dee758188a46a8734feb0bbb03fc6187625cc5842e815bf130399cf"}
Sep 10 19:20:42.917702 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.917665    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rkxv6" event={"ID":"8fa87b08-6c57-4461-a536-f2a9adeac807","Type":"ContainerStarted","Data":"851accf9dc194b557e510c186986582c8da9bef11b4042550ad3f1ea1c37fb10"}
Sep 10 19:20:42.918986 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.918961    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" event={"ID":"2f2e6b2f-afbe-45bb-84bc-582ace2e0379","Type":"ContainerStarted","Data":"1d37d04c7a2c0a49405e8d30a7c2a9d9ccd700e311ab596c1cdb1f29b50214d6"}
Sep 10 19:20:42.920199 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.920173    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67" event={"ID":"33fdd110-50e5-49e9-93f7-c57f7cfa5764","Type":"ContainerStarted","Data":"eac8d959f7ef82cb4d5b67684ab5429e687ca4992c656dcc95d5263403bdb834"}
Sep 10 19:20:42.921387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:42.921350    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d" event={"ID":"6097bb6c-42d7-434d-8fd2-e02d0514e025","Type":"ContainerStarted","Data":"7bac297994513b157c40522389de8df6d69922a94c6f2ebdfdded683f0664fe3"}
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.599290    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.599345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.599415    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.599480    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.599669    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.599738    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:45.599717799 +0000 UTC m=+38.539751614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : secret "router-metrics-certs-default" not found
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.600463    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.600523    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls podName:e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d nodeName:}" failed. No retries permitted until 2026-09-10 19:20:45.600505166 +0000 UTC m=+38.540538992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-sk5q6" (UID: "e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.600586    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.600597    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78875ff86d-blbk5: secret "image-registry-tls" not found
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.600633    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls podName:e7f4b902-9333-4b0e-bd8f-3c01620d0e79 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:45.600621689 +0000 UTC m=+38.540655503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls") pod "image-registry-78875ff86d-blbk5" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79") : secret "image-registry-tls" not found
Sep 10 19:20:43.600751 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.600711    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:45.60069266 +0000 UTC m=+38.540726485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : configmap references non-existent config key: service-ca.crt
Sep 10 19:20:43.689667 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.689629    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:20:43.690168 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.690148    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:20:43.694228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.690583    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:20:43.695179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.694721    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-plphg\""
Sep 10 19:20:43.695179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.694736    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:20:43.695179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.695043    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npjmj\""
Sep 10 19:20:43.695179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.695106    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:20:43.703952 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.703920    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:43.704084 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.704045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:43.704434 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:43.704121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:43.712222 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.712186    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:20:43.712369 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.712280    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls podName:ac4033cf-e6b0-4176-9105-4d945dc166f1 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:45.712258026 +0000 UTC m=+38.652291836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8dfpf" (UID: "ac4033cf-e6b0-4176-9105-4d945dc166f1") : secret "samples-operator-tls" not found
Sep 10 19:20:43.712850 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.712823    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:20:43.712965 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.712906    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert podName:04f372b4-a549-4f39-b87f-7f53a360a242 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:45.712874242 +0000 UTC m=+38.652908056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-wqgz2" (UID: "04f372b4-a549-4f39-b87f-7f53a360a242") : secret "networking-console-plugin-cert" not found
Sep 10 19:20:43.713089 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.713072    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:20:43.713164 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:43.713125    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls podName:c6f9c1d8-a78a-4541-bb95-e368edfee6fe nodeName:}" failed. No retries permitted until 2026-09-10 19:20:45.713110898 +0000 UTC m=+38.653144709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls") pod "dns-default-nh2g2" (UID: "c6f9c1d8-a78a-4541-bb95-e368edfee6fe") : secret "dns-default-metrics-tls" not found
Sep 10 19:20:44.939783 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:44.938499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn" event={"ID":"ccd4a7aa-770f-4dba-a34d-490ba52734b6","Type":"ContainerStarted","Data":"10ffdcd13cca653f93d31da75373254252f5db8dfc808dfe0dcbb22cecf353c6"}
Sep 10 19:20:44.955823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:44.955744    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-cv7hn" podStartSLOduration=34.25382684 podStartE2EDuration="35.95572522s" totalImagesPullingTime="1.70189838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:44.954318706 +0000 UTC m=+37.894352540" watchObservedRunningTime="2026-09-10 19:20:44.95572522 +0000 UTC m=+37.895759052"
Sep 10 19:20:45.077859 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.077820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:45.180271 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.180197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:45.628435 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.628393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:45.628710 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.628445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:45.628710 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.628488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:45.628710 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.628541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:45.628906 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.628713    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:20:45.628906 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.628775    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:49.628758589 +0000 UTC m=+42.568792414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : secret "router-metrics-certs-default" not found
Sep 10 19:20:45.629210 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.629187    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:45.629302 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.629237    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls podName:e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d nodeName:}" failed. No retries permitted until 2026-09-10 19:20:49.629222692 +0000 UTC m=+42.569256503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-sk5q6" (UID: "e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:45.629361 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.629311    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:20:45.629361 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.629323    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78875ff86d-blbk5: secret "image-registry-tls" not found
Sep 10 19:20:45.629361 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.629355    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls podName:e7f4b902-9333-4b0e-bd8f-3c01620d0e79 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:49.629344325 +0000 UTC m=+42.569378139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls") pod "image-registry-78875ff86d-blbk5" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79") : secret "image-registry-tls" not found
Sep 10 19:20:45.629516 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.629415    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:49.629405593 +0000 UTC m=+42.569439407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : configmap references non-existent config key: service-ca.crt
Sep 10 19:20:45.730010 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.729959    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:45.730195 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.730110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:45.730252 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.730207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:45.730374 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.730355    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:20:45.730463 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.730394    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:20:45.730463 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.730426    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert podName:04f372b4-a549-4f39-b87f-7f53a360a242 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:49.730405513 +0000 UTC m=+42.670439330 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-wqgz2" (UID: "04f372b4-a549-4f39-b87f-7f53a360a242") : secret "networking-console-plugin-cert" not found
Sep 10 19:20:45.730561 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.730467    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls podName:c6f9c1d8-a78a-4541-bb95-e368edfee6fe nodeName:}" failed. No retries permitted until 2026-09-10 19:20:49.730447644 +0000 UTC m=+42.670481476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls") pod "dns-default-nh2g2" (UID: "c6f9c1d8-a78a-4541-bb95-e368edfee6fe") : secret "dns-default-metrics-tls" not found
Sep 10 19:20:45.730561 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.730473    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:20:45.730561 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:45.730529    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls podName:ac4033cf-e6b0-4176-9105-4d945dc166f1 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:49.730512786 +0000 UTC m=+42.670546612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8dfpf" (UID: "ac4033cf-e6b0-4176-9105-4d945dc166f1") : secret "samples-operator-tls" not found
Sep 10 19:20:45.876448 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.876411    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:45.943601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:45.943565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:46.527630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:46.527593    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:46.820229 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:46.820143    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:49.000129 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.000090    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:49.100442 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.100402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:49.669906 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.669856    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:49.670102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.669942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:49.670102 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670013    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:49.670102 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670058    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:20:49.670102 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670077    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78875ff86d-blbk5: secret "image-registry-tls" not found
Sep 10 19:20:49.670102 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670081    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls podName:e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.670063886 +0000 UTC m=+50.610097720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-sk5q6" (UID: "e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:49.670345 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670124    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls podName:e7f4b902-9333-4b0e-bd8f-3c01620d0e79 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.670105842 +0000 UTC m=+50.610139669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls") pod "image-registry-78875ff86d-blbk5" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79") : secret "image-registry-tls" not found
Sep 10 19:20:49.670345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.670144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:49.670345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.670199    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:49.670345 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670269    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:20:49.670345 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670296    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.670279598 +0000 UTC m=+50.610313426 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : configmap references non-existent config key: service-ca.crt
Sep 10 19:20:49.670345 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.670324    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.670312088 +0000 UTC m=+50.610346119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : secret "router-metrics-certs-default" not found
Sep 10 19:20:49.771486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.771442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:49.771757 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.771518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:49.771757 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:49.771621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:49.771757 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.771630    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:20:49.771757 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.771708    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls podName:c6f9c1d8-a78a-4541-bb95-e368edfee6fe nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.771685537 +0000 UTC m=+50.711719372 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls") pod "dns-default-nh2g2" (UID: "c6f9c1d8-a78a-4541-bb95-e368edfee6fe") : secret "dns-default-metrics-tls" not found
Sep 10 19:20:49.771757 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.771717    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:20:49.771757 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.771754    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:20:49.772077 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.771783    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert podName:04f372b4-a549-4f39-b87f-7f53a360a242 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.771759093 +0000 UTC m=+50.711792922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-wqgz2" (UID: "04f372b4-a549-4f39-b87f-7f53a360a242") : secret "networking-console-plugin-cert" not found
Sep 10 19:20:49.772077 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:49.771801    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls podName:ac4033cf-e6b0-4176-9105-4d945dc166f1 nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.771793115 +0000 UTC m=+50.711826925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8dfpf" (UID: "ac4033cf-e6b0-4176-9105-4d945dc166f1") : secret "samples-operator-tls" not found
Sep 10 19:20:50.041444 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:50.041345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:52.906814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.906764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:52.968206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.968094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh" event={"ID":"1a7b5479-66fb-4a0a-83d6-1a50656b07af","Type":"ContainerStarted","Data":"9a83b49a873a3d992dcfe1c11b1bb5f594e20eafe73b3df479fc046f02823459"}
Sep 10 19:20:52.970720 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.970284    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w" event={"ID":"20d869f4-b859-45f8-a21d-d90bf157087f","Type":"ContainerStarted","Data":"78d779a971c1635e1ed2c786796e29ed4029f86da3dd90a1168611dc282cd577"}
Sep 10 19:20:52.973458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.972999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rkxv6" event={"ID":"8fa87b08-6c57-4461-a536-f2a9adeac807","Type":"ContainerStarted","Data":"2ca06360e09215507b47015bd924a7761fcc334041d5893711fcb3fd028e22e1"}
Sep 10 19:20:52.975864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.975263    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/0.log"
Sep 10 19:20:52.975864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.975297    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f2e6b2f-afbe-45bb-84bc-582ace2e0379" containerID="3c732bd32e19a75b354e2a168a11ab79a17ff01dba86fa56160e8e4340878a4a" exitCode=255
Sep 10 19:20:52.975864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.975499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" event={"ID":"2f2e6b2f-afbe-45bb-84bc-582ace2e0379","Type":"ContainerDied","Data":"3c732bd32e19a75b354e2a168a11ab79a17ff01dba86fa56160e8e4340878a4a"}
Sep 10 19:20:52.975864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.975612    2562 scope.go:117] "RemoveContainer" containerID="3c732bd32e19a75b354e2a168a11ab79a17ff01dba86fa56160e8e4340878a4a"
Sep 10 19:20:52.977604 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.977553    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67" event={"ID":"33fdd110-50e5-49e9-93f7-c57f7cfa5764","Type":"ContainerStarted","Data":"4a78fdd1a2ebf0dcf9e150d53657c330815f20b54266193dd0c6f07e0eb3b4a6"}
Sep 10 19:20:52.980491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.980465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:52.981289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.981263    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b3d75eb-156b-48b4-b39c-68e9907810a5" containerID="cb29306e4b106a1edb5cd1bbfcb9b99ea93c89e71830ec0ad969883c116ea813" exitCode=0
Sep 10 19:20:52.981399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.981341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerDied","Data":"cb29306e4b106a1edb5cd1bbfcb9b99ea93c89e71830ec0ad969883c116ea813"}
Sep 10 19:20:52.983097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.983074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d" event={"ID":"6097bb6c-42d7-434d-8fd2-e02d0514e025","Type":"ContainerStarted","Data":"85ac32719eaef9367d12c531e5b11c8ad8a8ac0d2b6ef4b5c617ff66340af124"}
Sep 10 19:20:52.986898 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:52.986842    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh" podStartSLOduration=34.142445509 podStartE2EDuration="43.986825619s" totalImagesPullingTime="9.84438011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:52.986383546 +0000 UTC m=+45.926417380" watchObservedRunningTime="2026-09-10 19:20:52.986825619 +0000 UTC m=+45.926859453"
Sep 10 19:20:53.011948 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.011870    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67" podStartSLOduration=34.145261365 podStartE2EDuration="44.01184943s" totalImagesPullingTime="9.866588065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:53.010604974 +0000 UTC m=+45.950638808" watchObservedRunningTime="2026-09-10 19:20:53.01184943 +0000 UTC m=+45.951883264"
Sep 10 19:20:53.055848 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.055787    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rkxv6" podStartSLOduration=2.279334096 podStartE2EDuration="12.05576251s" totalImagesPullingTime="9.776428414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:53.036250806 +0000 UTC m=+45.976284641" watchObservedRunningTime="2026-09-10 19:20:53.05576251 +0000 UTC m=+45.995796344"
Sep 10 19:20:53.075402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.074347    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-rzn7d" podStartSLOduration=34.254423273 podStartE2EDuration="44.074324697s" totalImagesPullingTime="9.819901424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:53.072805981 +0000 UTC m=+46.012839815" watchObservedRunningTime="2026-09-10 19:20:53.074324697 +0000 UTC m=+46.014358525"
Sep 10 19:20:53.530018 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.529920    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:53.573969 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.571909    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w" podStartSLOduration=34.680699526 podStartE2EDuration="44.571868467s" totalImagesPullingTime="9.891168941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:53.13079366 +0000 UTC m=+46.070827497" watchObservedRunningTime="2026-09-10 19:20:53.571868467 +0000 UTC m=+46.511902301"
Sep 10 19:20:53.573969 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.572466    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cn5vq"]
Sep 10 19:20:53.594492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.594461    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cn5vq"]
Sep 10 19:20:53.594679 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.594596    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.598422 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.598131    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 19:20:53.598422 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.598210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 19:20:53.600316 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.600287    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7hkk9\""
Sep 10 19:20:53.608257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.608214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:53.712303 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.712262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1bb7defd-3c95-44ee-956c-6e6e567622aa-crio-socket\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.712487 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.712379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjpsq\" (UniqueName: \"kubernetes.io/projected/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-api-access-cjpsq\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.712487 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.712456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.712567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.712508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bb7defd-3c95-44ee-956c-6e6e567622aa-data-volume\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.712607 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.712579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.813739 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.813627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1bb7defd-3c95-44ee-956c-6e6e567622aa-crio-socket\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.813739 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.813724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjpsq\" (UniqueName: \"kubernetes.io/projected/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-api-access-cjpsq\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.813999 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.813772    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.813999 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.813823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bb7defd-3c95-44ee-956c-6e6e567622aa-data-volume\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.813999 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.813858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.813999 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.813912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1bb7defd-3c95-44ee-956c-6e6e567622aa-crio-socket\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.813999 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:53.813961    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:53.813999 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:53.813993    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:20:53.814289 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:53.814040    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:20:54.31401736 +0000 UTC m=+47.254051178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:53.814289 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:53.814060    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:20:54.314050387 +0000 UTC m=+47.254084204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : configmap "kube-rbac-proxy" not found
Sep 10 19:20:53.814289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.814219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bb7defd-3c95-44ee-956c-6e6e567622aa-data-volume\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.824165 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.824131    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjpsq\" (UniqueName: \"kubernetes.io/projected/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-api-access-cjpsq\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:53.989446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.989417    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:20:53.989933 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.989863    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/0.log"
Sep 10 19:20:53.989933 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.989915    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f2e6b2f-afbe-45bb-84bc-582ace2e0379" containerID="9032ac66d6274c753e14935accdcf0f7a3ac0eb4be5f62427c07c78269321ba7" exitCode=255
Sep 10 19:20:53.990059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.990016    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" event={"ID":"2f2e6b2f-afbe-45bb-84bc-582ace2e0379","Type":"ContainerDied","Data":"9032ac66d6274c753e14935accdcf0f7a3ac0eb4be5f62427c07c78269321ba7"}
Sep 10 19:20:53.990116 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.990105    2562 scope.go:117] "RemoveContainer" containerID="3c732bd32e19a75b354e2a168a11ab79a17ff01dba86fa56160e8e4340878a4a"
Sep 10 19:20:53.990933 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.990470    2562 scope.go:117] "RemoveContainer" containerID="9032ac66d6274c753e14935accdcf0f7a3ac0eb4be5f62427c07c78269321ba7"
Sep 10 19:20:53.990933 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:53.990692    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-7b5pk_openshift-console-operator(2f2e6b2f-afbe-45bb-84bc-582ace2e0379)\"" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" podUID="2f2e6b2f-afbe-45bb-84bc-582ace2e0379"
Sep 10 19:20:53.994640 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.994606    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b3d75eb-156b-48b4-b39c-68e9907810a5" containerID="99253a5807063de1bb9e4d1736a078b1f523e330e41b4809de66adc5d60f1105" exitCode=0
Sep 10 19:20:53.995287 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:53.995261    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerDied","Data":"99253a5807063de1bb9e4d1736a078b1f523e330e41b4809de66adc5d60f1105"}
Sep 10 19:20:54.078405 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:54.078227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:54.194787 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:54.194755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:54.295695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:54.295659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:54.318289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:54.318254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:54.318474 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:54.318312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:54.318474 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:54.318417    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:20:54.318474 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:54.318453    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:54.318474 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:54.318472    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:20:55.318457491 +0000 UTC m=+48.258491302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : configmap "kube-rbac-proxy" not found
Sep 10 19:20:54.318718 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:54.318521    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:20:55.318501142 +0000 UTC m=+48.258534953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:54.600450 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:54.600416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:54.703645 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:54.703612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:55.003811 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.003783    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:20:55.004259 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.004132    2562 scope.go:117] "RemoveContainer" containerID="9032ac66d6274c753e14935accdcf0f7a3ac0eb4be5f62427c07c78269321ba7"
Sep 10 19:20:55.004337 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:55.004309    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-7b5pk_openshift-console-operator(2f2e6b2f-afbe-45bb-84bc-582ace2e0379)\"" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" podUID="2f2e6b2f-afbe-45bb-84bc-582ace2e0379"
Sep 10 19:20:55.007187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.007166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" event={"ID":"2b3d75eb-156b-48b4-b39c-68e9907810a5","Type":"ContainerStarted","Data":"1fce72789ab23b39c28002f9f9dc8a088ae09e4eb1c6b52e7fa7cbac1eb0f04c"}
Sep 10 19:20:55.032487 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.032454    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:55.048394 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.048334    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mm5gp" podStartSLOduration=8.787096604 podStartE2EDuration="47.048315133s" totalImagesPullingTime="38.261218529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:55.047551934 +0000 UTC m=+47.987585806" watchObservedRunningTime="2026-09-10 19:20:55.048315133 +0000 UTC m=+47.988348966"
Sep 10 19:20:55.172520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.172472    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:55.231780 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.231743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:55.316881 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.316797    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:55.327762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.327729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:55.327879 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.327805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:55.327942 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:55.327904    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:55.327942 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:55.327911    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:20:55.328015 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:55.327978    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.327958351 +0000 UTC m=+50.267992169 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:55.328015 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:55.327994    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:20:57.327987083 +0000 UTC m=+50.268020896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : configmap "kube-rbac-proxy" not found
Sep 10 19:20:55.350456 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.350436    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2htct_491040f7-5239-4e74-ac39-1b5a8190e333/dns-node-resolver/0.log"
Sep 10 19:20:55.409533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:55.409486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:56.550402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:56.550370    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bhgv7_05dde958-85c8-4751-87d7-cd7b3d9314cc/node-ca/0.log"
Sep 10 19:20:56.947622 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:56.947589    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:57.006517 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.006463    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:57.348029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.347938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:57.348029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.347998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:20:57.348230 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.348100    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:57.348230 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.348172    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:21:01.348156098 +0000 UTC m=+54.288189914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:20:57.348230 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.348107    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:20:57.348230 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.348231    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:21:01.348218598 +0000 UTC m=+54.288252409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : configmap "kube-rbac-proxy" not found
Sep 10 19:20:57.350028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.350007    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rkxv6_8fa87b08-6c57-4461-a536-f2a9adeac807/serve-healthcheck-canary/0.log"
Sep 10 19:20:57.752470 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.752423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:20:57.752470 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.752475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.752517    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.752565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752565    2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752629    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls podName:e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.75261325 +0000 UTC m=+66.692647060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-sk5q6" (UID: "e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752673    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752734    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.7527175 +0000 UTC m=+66.692751333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : secret "router-metrics-certs-default" not found
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752678    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752752    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78875ff86d-blbk5: secret "image-registry-tls" not found
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752755    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle podName:fc444d45-54f7-4c61-a08d-34658072f79a nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.752743836 +0000 UTC m=+66.692777651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle") pod "router-default-9dc57874b-zzp9h" (UID: "fc444d45-54f7-4c61-a08d-34658072f79a") : configmap references non-existent config key: service-ca.crt
Sep 10 19:20:57.752974 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.752779    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls podName:e7f4b902-9333-4b0e-bd8f-3c01620d0e79 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.752771125 +0000 UTC m=+66.692804937 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls") pod "image-registry-78875ff86d-blbk5" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79") : secret "image-registry-tls" not found
Sep 10 19:20:57.853927 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.853871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:20:57.854088 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.853985    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:20:57.854088 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.854042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:20:57.854088 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.854043    2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:20:57.854185 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.854122    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls podName:ac4033cf-e6b0-4176-9105-4d945dc166f1 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.854102908 +0000 UTC m=+66.794136719 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8dfpf" (UID: "ac4033cf-e6b0-4176-9105-4d945dc166f1") : secret "samples-operator-tls" not found
Sep 10 19:20:57.854185 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.854120    2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:20:57.854185 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.854129    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:20:57.854185 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.854149    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert podName:04f372b4-a549-4f39-b87f-7f53a360a242 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.854143547 +0000 UTC m=+66.794177358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-wqgz2" (UID: "04f372b4-a549-4f39-b87f-7f53a360a242") : secret "networking-console-plugin-cert" not found
Sep 10 19:20:57.854185 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:20:57.854179    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls podName:c6f9c1d8-a78a-4541-bb95-e368edfee6fe nodeName:}" failed. No retries permitted until 2026-09-10 19:21:13.854165776 +0000 UTC m=+66.794199588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls") pod "dns-default-nh2g2" (UID: "c6f9c1d8-a78a-4541-bb95-e368edfee6fe") : secret "dns-default-metrics-tls" not found
Sep 10 19:20:57.922547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:57.922498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:20:58.522192 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:20:58.522154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:01.176816 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:01.176785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:01.244883 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:01.244847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:01.390467 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:01.390429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:21:01.390642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:01.390489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:21:01.390642 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:01.390601    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:21:01.390775 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:01.390694    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls podName:1bb7defd-3c95-44ee-956c-6e6e567622aa nodeName:}" failed. No retries permitted until 2026-09-10 19:21:09.390672114 +0000 UTC m=+62.330705948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cn5vq" (UID: "1bb7defd-3c95-44ee-956c-6e6e567622aa") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:21:01.391069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:01.391046    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1bb7defd-3c95-44ee-956c-6e6e567622aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:21:01.949826 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:01.949789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:02.123757 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:02.123717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:02.198329 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:02.198303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:02.295022 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:02.294922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:21:02.295022 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:02.294997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:21:02.295453 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:02.295437    2562 scope.go:117] "RemoveContainer" containerID="9032ac66d6274c753e14935accdcf0f7a3ac0eb4be5f62427c07c78269321ba7"
Sep 10 19:21:02.295643 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:02.295621    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-7b5pk_openshift-console-operator(2f2e6b2f-afbe-45bb-84bc-582ace2e0379)\"" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" podUID="2f2e6b2f-afbe-45bb-84bc-582ace2e0379"
Sep 10 19:21:02.341215 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:02.341181    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:03.325280 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:03.325246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:03.528044 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:03.528007    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:03.716876 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:03.716841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:03.919240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:03.919207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:04.521262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:04.521232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:04.726093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:04.726061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.051660 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:05.051622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.152266 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:05.152233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.534090 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:05.534053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.646657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:05.646624    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:05.927698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:05.927668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:06.349761 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:06.349682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:06.904152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:06.904123    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wk98n"
Sep 10 19:21:06.916450 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:06.916419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:06.982549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:06.982518    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:07.317005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:07.316924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:07.935586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:07.935544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:07.992286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:07.992246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:08.314561 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:08.314444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:08.728512 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:08.728478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:09.463712 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.463670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:21:09.465991 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.465958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1bb7defd-3c95-44ee-956c-6e6e567622aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cn5vq\" (UID: \"1bb7defd-3c95-44ee-956c-6e6e567622aa\") " pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:21:09.507000 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.506969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:09.509383 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.509366    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7hkk9\""
Sep 10 19:21:09.517550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.517527    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cn5vq"
Sep 10 19:21:09.577477 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.577447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:09.730941 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.730840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:09.787087 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:09.787051    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cn5vq"]
Sep 10 19:21:09.789940 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:09.789906    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb7defd_3c95_44ee_956c_6e6e567622aa.slice/crio-35b7c66901fe1a6a6987e2862e2d7e8ff4a44f3727575b5f1e535c7da65750c1 WatchSource:0}: Error finding container 35b7c66901fe1a6a6987e2862e2d7e8ff4a44f3727575b5f1e535c7da65750c1: Status 404 returned error can't find the container with id 35b7c66901fe1a6a6987e2862e2d7e8ff4a44f3727575b5f1e535c7da65750c1
Sep 10 19:21:10.056403 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:10.056318    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cn5vq" event={"ID":"1bb7defd-3c95-44ee-956c-6e6e567622aa","Type":"ContainerStarted","Data":"bd9a55dbef7cc74dc610678f3d718c90553295ca3e51553c2f28855d5656b2d0"}
Sep 10 19:21:10.056403 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:10.056378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cn5vq" event={"ID":"1bb7defd-3c95-44ee-956c-6e6e567622aa","Type":"ContainerStarted","Data":"35b7c66901fe1a6a6987e2862e2d7e8ff4a44f3727575b5f1e535c7da65750c1"}
Sep 10 19:21:10.121490 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:10.121452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:10.722298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:10.722258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:11.061358 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:11.061322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cn5vq" event={"ID":"1bb7defd-3c95-44ee-956c-6e6e567622aa","Type":"ContainerStarted","Data":"95ef7a9cea2caad99f392f8a8921c190279b3f113f10b9fa17ac7c8401bc2665"}
Sep 10 19:21:11.126476 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:11.126445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:11.325863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:11.325778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:11.521845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:11.521811    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:13.070242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.070202    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cn5vq" event={"ID":"1bb7defd-3c95-44ee-956c-6e6e567622aa","Type":"ContainerStarted","Data":"221caa7290acf14fbb4565439d501af46e0896c1e8f1249c0425db5d5fd57774"}
Sep 10 19:21:13.091293 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.091212    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cn5vq" podStartSLOduration=17.847184837 podStartE2EDuration="20.091190151s" totalImagesPullingTime="2.244005314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:13.089089507 +0000 UTC m=+66.029123340" watchObservedRunningTime="2026-09-10 19:21:13.091190151 +0000 UTC m=+66.031224052"
Sep 10 19:21:13.298624 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.298570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:21:13.298624 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.298639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:21:13.301412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.301391    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:21:13.301508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.301438    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:21:13.311695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.311668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3338777-3059-4b20-b06a-134279c5ad91-metrics-certs\") pod \"network-metrics-daemon-slrvh\" (UID: \"f3338777-3059-4b20-b06a-134279c5ad91\") " pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:21:13.311781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.311757    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e50d4c91-53ee-4496-9854-3a3abb0a1ca7-original-pull-secret\") pod \"global-pull-secret-syncer-nhl6m\" (UID: \"e50d4c91-53ee-4496-9854-3a3abb0a1ca7\") " pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:21:13.399265 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.399219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:21:13.401656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.401633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krw9g\" (UniqueName: \"kubernetes.io/projected/8a327e33-4bc2-47ae-b268-c8225e73f304-kube-api-access-krw9g\") pod \"network-check-target-xgn2n\" (UID: \"8a327e33-4bc2-47ae-b268-c8225e73f304\") " pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:21:13.423258 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.423222    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhl6m"
Sep 10 19:21:13.435928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.435900    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-plphg\""
Sep 10 19:21:13.443203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.443175    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:21:13.445916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.445880    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npjmj\""
Sep 10 19:21:13.454152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.454132    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-slrvh"
Sep 10 19:21:13.698291 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.698210    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nhl6m"]
Sep 10 19:21:13.700951 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:13.700925    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50d4c91_53ee_4496_9854_3a3abb0a1ca7.slice/crio-b36a12f77dc70341fcf9ed878cba606ef6efb78d677ec92c09ccc969f6095435 WatchSource:0}: Error finding container b36a12f77dc70341fcf9ed878cba606ef6efb78d677ec92c09ccc969f6095435: Status 404 returned error can't find the container with id b36a12f77dc70341fcf9ed878cba606ef6efb78d677ec92c09ccc969f6095435
Sep 10 19:21:13.724002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.723977    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xgn2n"]
Sep 10 19:21:13.726533 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:13.726497    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a327e33_4bc2_47ae_b268_c8225e73f304.slice/crio-f322e7e439bd28dc8aaa52a4cff26807e7719efdd55ccdffe08213e7baed16aa WatchSource:0}: Error finding container f322e7e439bd28dc8aaa52a4cff26807e7719efdd55ccdffe08213e7baed16aa: Status 404 returned error can't find the container with id f322e7e439bd28dc8aaa52a4cff26807e7719efdd55ccdffe08213e7baed16aa
Sep 10 19:21:13.747624 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.747604    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-slrvh"]
Sep 10 19:21:13.764176 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:13.764148    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3338777_3059_4b20_b06a_134279c5ad91.slice/crio-a68d296830c225ecd46905da77bb1785d00753f404a74311da5a65b30a792750 WatchSource:0}: Error finding container a68d296830c225ecd46905da77bb1785d00753f404a74311da5a65b30a792750: Status 404 returned error can't find the container with id a68d296830c225ecd46905da77bb1785d00753f404a74311da5a65b30a792750
Sep 10 19:21:13.802979 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.802951    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:13.803117 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.803015    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:21:13.803117 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.803050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:21:13.803117 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.803083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:13.803765 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.803741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc444d45-54f7-4c61-a08d-34658072f79a-service-ca-bundle\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:13.805331 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.805313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-sk5q6\" (UID: \"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:21:13.805746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.805721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"image-registry-78875ff86d-blbk5\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") " pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:21:13.805825 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.805741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc444d45-54f7-4c61-a08d-34658072f79a-metrics-certs\") pod \"router-default-9dc57874b-zzp9h\" (UID: \"fc444d45-54f7-4c61-a08d-34658072f79a\") " pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:13.903685 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.903647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:21:13.903874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.903705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:21:13.903874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.903738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:21:13.906154 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.906127    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6f9c1d8-a78a-4541-bb95-e368edfee6fe-metrics-tls\") pod \"dns-default-nh2g2\" (UID: \"c6f9c1d8-a78a-4541-bb95-e368edfee6fe\") " pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:21:13.906154 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.906143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ac4033cf-e6b0-4176-9105-4d945dc166f1-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8dfpf\" (UID: \"ac4033cf-e6b0-4176-9105-4d945dc166f1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:21:13.906290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:13.906144    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04f372b4-a549-4f39-b87f-7f53a360a242-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-wqgz2\" (UID: \"04f372b4-a549-4f39-b87f-7f53a360a242\") " pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:21:14.001170 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.001089    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x5vxr\""
Sep 10 19:21:14.009299 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.009274    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:21:14.022224 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.022188    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kfzpl\""
Sep 10 19:21:14.029368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.029341    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"
Sep 10 19:21:14.029857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.029831    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qfppk\""
Sep 10 19:21:14.038416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.038387    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:14.075444 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.075258    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cqwnz\""
Sep 10 19:21:14.079549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.078144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-slrvh" event={"ID":"f3338777-3059-4b20-b06a-134279c5ad91","Type":"ContainerStarted","Data":"a68d296830c225ecd46905da77bb1785d00753f404a74311da5a65b30a792750"}
Sep 10 19:21:14.079549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.079535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nhl6m" event={"ID":"e50d4c91-53ee-4496-9854-3a3abb0a1ca7","Type":"ContainerStarted","Data":"b36a12f77dc70341fcf9ed878cba606ef6efb78d677ec92c09ccc969f6095435"}
Sep 10 19:21:14.081580 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.081554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xgn2n" event={"ID":"8a327e33-4bc2-47ae-b268-c8225e73f304","Type":"ContainerStarted","Data":"b3d97178cc25435409bae2ae4ee7aafb56c77304ffee8ab11890c8ad10d675d6"}
Sep 10 19:21:14.081666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.081598    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xgn2n" event={"ID":"8a327e33-4bc2-47ae-b268-c8225e73f304","Type":"ContainerStarted","Data":"f322e7e439bd28dc8aaa52a4cff26807e7719efdd55ccdffe08213e7baed16aa"}
Sep 10 19:21:14.083577 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.083556    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2"
Sep 10 19:21:14.106831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.106806    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2kts8\""
Sep 10 19:21:14.111239 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.111209    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ss72r\""
Sep 10 19:21:14.115464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.115411    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:21:14.118992 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.118971    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"
Sep 10 19:21:14.289988 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.289827    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xgn2n" podStartSLOduration=66.289806112 podStartE2EDuration="1m6.289806112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:14.099220595 +0000 UTC m=+67.039254430" watchObservedRunningTime="2026-09-10 19:21:14.289806112 +0000 UTC m=+67.229840021"
Sep 10 19:21:14.290767 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.290739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78875ff86d-blbk5"]
Sep 10 19:21:14.293987 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:14.293956    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f4b902_9333_4b0e_bd8f_3c01620d0e79.slice/crio-f18bfba335f81c1951470a41a98cbcea7a0406524da02d99bc60cd7734a25031 WatchSource:0}: Error finding container f18bfba335f81c1951470a41a98cbcea7a0406524da02d99bc60cd7734a25031: Status 404 returned error can't find the container with id f18bfba335f81c1951470a41a98cbcea7a0406524da02d99bc60cd7734a25031
Sep 10 19:21:14.311113 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.311082    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6"]
Sep 10 19:21:14.314492 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:14.314461    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7746c37_e94a_4efd_a1f9_e4ad8e13ff9d.slice/crio-b27338215b1e2fa607012efe3ee3ce09f445cdca6fa1b14d1e03e94d486abb5e WatchSource:0}: Error finding container b27338215b1e2fa607012efe3ee3ce09f445cdca6fa1b14d1e03e94d486abb5e: Status 404 returned error can't find the container with id b27338215b1e2fa607012efe3ee3ce09f445cdca6fa1b14d1e03e94d486abb5e
Sep 10 19:21:14.342740 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.342457    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9dc57874b-zzp9h"]
Sep 10 19:21:14.345607 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:14.345569    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc444d45_54f7_4c61_a08d_34658072f79a.slice/crio-edde500dfd6f85dc6d977f6cf077e8b88a92a72bd472f412358e7aae26e87939 WatchSource:0}: Error finding container edde500dfd6f85dc6d977f6cf077e8b88a92a72bd472f412358e7aae26e87939: Status 404 returned error can't find the container with id edde500dfd6f85dc6d977f6cf077e8b88a92a72bd472f412358e7aae26e87939
Sep 10 19:21:14.371868 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.371842    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-wqgz2"]
Sep 10 19:21:14.375216 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:14.375177    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f372b4_a549_4f39_b87f_7f53a360a242.slice/crio-aa91c8b19f3ca45889d0df430d79cf648a9e5048c6835643b13fcab3f71d5ac1 WatchSource:0}: Error finding container aa91c8b19f3ca45889d0df430d79cf648a9e5048c6835643b13fcab3f71d5ac1: Status 404 returned error can't find the container with id aa91c8b19f3ca45889d0df430d79cf648a9e5048c6835643b13fcab3f71d5ac1
Sep 10 19:21:14.404250 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.404001    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nh2g2"]
Sep 10 19:21:14.406802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.406728    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf"]
Sep 10 19:21:14.411085 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:14.410967    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f9c1d8_a78a_4541_bb95_e368edfee6fe.slice/crio-6d611e55a07b37c0feef9d51820b7017e0fd2ada5aeae02301bb640412370e10 WatchSource:0}: Error finding container 6d611e55a07b37c0feef9d51820b7017e0fd2ada5aeae02301bb640412370e10: Status 404 returned error can't find the container with id 6d611e55a07b37c0feef9d51820b7017e0fd2ada5aeae02301bb640412370e10
Sep 10 19:21:14.689705 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:14.689667    2562 scope.go:117] "RemoveContainer" containerID="9032ac66d6274c753e14935accdcf0f7a3ac0eb4be5f62427c07c78269321ba7"
Sep 10 19:21:15.090846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.090809    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78875ff86d-blbk5" event={"ID":"e7f4b902-9333-4b0e-bd8f-3c01620d0e79","Type":"ContainerStarted","Data":"ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d"}
Sep 10 19:21:15.091294 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.090879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78875ff86d-blbk5" event={"ID":"e7f4b902-9333-4b0e-bd8f-3c01620d0e79","Type":"ContainerStarted","Data":"f18bfba335f81c1951470a41a98cbcea7a0406524da02d99bc60cd7734a25031"}
Sep 10 19:21:15.091733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.091690    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:21:15.101241 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.100153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-slrvh" event={"ID":"f3338777-3059-4b20-b06a-134279c5ad91","Type":"ContainerStarted","Data":"2bce8929c0ed2e10b14e5fe7de31a4cdc755c35a6c22c04d7f043ae33d38565e"}
Sep 10 19:21:15.106112 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.106047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf" event={"ID":"ac4033cf-e6b0-4176-9105-4d945dc166f1","Type":"ContainerStarted","Data":"7cdbf49e38b3cb1eb59d18ac0bec80176c30435f387935932e7e84d6d4a8f20f"}
Sep 10 19:21:15.108432 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.108366    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nh2g2" event={"ID":"c6f9c1d8-a78a-4541-bb95-e368edfee6fe","Type":"ContainerStarted","Data":"6d611e55a07b37c0feef9d51820b7017e0fd2ada5aeae02301bb640412370e10"}
Sep 10 19:21:15.111127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.111004    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2" event={"ID":"04f372b4-a549-4f39-b87f-7f53a360a242","Type":"ContainerStarted","Data":"aa91c8b19f3ca45889d0df430d79cf648a9e5048c6835643b13fcab3f71d5ac1"}
Sep 10 19:21:15.113665 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.113613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6" event={"ID":"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d","Type":"ContainerStarted","Data":"b27338215b1e2fa607012efe3ee3ce09f445cdca6fa1b14d1e03e94d486abb5e"}
Sep 10 19:21:15.120992 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.120175    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:21:15.120992 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.120270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" event={"ID":"2f2e6b2f-afbe-45bb-84bc-582ace2e0379","Type":"ContainerStarted","Data":"892f75a436f2890efc67df134da8a165ab3c9f8652a3c5d10dc0e506b7ac2e16"}
Sep 10 19:21:15.121212 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.121048    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:21:15.126689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.126583    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9dc57874b-zzp9h" event={"ID":"fc444d45-54f7-4c61-a08d-34658072f79a","Type":"ContainerStarted","Data":"a833198d4dbae3715707ebefc0dfb1663eff1b955efaa796ac40ab7905228dd5"}
Sep 10 19:21:15.126689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.126627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9dc57874b-zzp9h" event={"ID":"fc444d45-54f7-4c61-a08d-34658072f79a","Type":"ContainerStarted","Data":"edde500dfd6f85dc6d977f6cf077e8b88a92a72bd472f412358e7aae26e87939"}
Sep 10 19:21:15.126689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.126652    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:21:15.138084 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.137547    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-7b5pk" podStartSLOduration=56.363085218 podStartE2EDuration="1m6.137526887s" totalImagesPullingTime="9.774441669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:15.137024776 +0000 UTC m=+68.077058633" watchObservedRunningTime="2026-09-10 19:21:15.137526887 +0000 UTC m=+68.077560722"
Sep 10 19:21:15.146651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.143056    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78875ff86d-blbk5" podStartSLOduration=67.143034775 podStartE2EDuration="1m7.143034775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:15.11722736 +0000 UTC m=+68.057261206" watchObservedRunningTime="2026-09-10 19:21:15.143034775 +0000 UTC m=+68.083068609"
Sep 10 19:21:15.160746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.160655    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9dc57874b-zzp9h" podStartSLOduration=66.160632395 podStartE2EDuration="1m6.160632395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:15.160502442 +0000 UTC m=+68.100536276" watchObservedRunningTime="2026-09-10 19:21:15.160632395 +0000 UTC m=+68.100666219"
Sep 10 19:21:15.669349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.669282    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-7b5pk"
Sep 10 19:21:15.870081 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.869995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:15.969866 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:15.969649    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.029494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.029456    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.039531 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.039452    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:16.042610 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.042396    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:16.137284 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.137236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-slrvh" event={"ID":"f3338777-3059-4b20-b06a-134279c5ad91","Type":"ContainerStarted","Data":"42a7385db9f4be2c1dcfcc37d6289af64e8abfc4f14e0e72963e62dfb07bce50"}
Sep 10 19:21:16.138418 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.138368    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:16.140068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.140044    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9dc57874b-zzp9h"
Sep 10 19:21:16.150403 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.150330    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.155935 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.154689    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-slrvh" podStartSLOduration=67.061404086 podStartE2EDuration="1m8.154669181s" totalImagesPullingTime="1.093265095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:16.154475621 +0000 UTC m=+69.094509477" watchObservedRunningTime="2026-09-10 19:21:16.154669181 +0000 UTC m=+69.094703017"
Sep 10 19:21:16.525403 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.525281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.586007 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.585919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.924117 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.924082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:16.981098 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:16.981058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:17.142139 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:17.142082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2" event={"ID":"04f372b4-a549-4f39-b87f-7f53a360a242","Type":"ContainerStarted","Data":"d2d27d83c665f136e947de472f98355d4034bdc7434fb616922c329a53d69adb"}
Sep 10 19:21:17.158509 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:17.158452    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-wqgz2" podStartSLOduration=39.376349157 podStartE2EDuration="41.158431928s" totalImagesPullingTime="1.782082771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:17.158146125 +0000 UTC m=+70.098179957" watchObservedRunningTime="2026-09-10 19:21:17.158431928 +0000 UTC m=+70.098465762"
Sep 10 19:21:17.326213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:17.326136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:17.392804 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:17.392772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:17.735432 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:17.735396    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:17.938612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:17.938565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:18.557419 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:18.557373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:18.659635 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:18.659573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:18.758967 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:18.758933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:19.969354 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:19.969307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:20.157346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.157311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf" event={"ID":"ac4033cf-e6b0-4176-9105-4d945dc166f1","Type":"ContainerStarted","Data":"de2858d1045fb29716dda201efdf4f45e7fcd376c34a50438e39aaec7dabcab0"}
Sep 10 19:21:20.157530 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.157364    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf" event={"ID":"ac4033cf-e6b0-4176-9105-4d945dc166f1","Type":"ContainerStarted","Data":"5a747d1cbc3f0b2d868a2f21aa01edb39db8047573f86c1251cab5223e4c0083"}
Sep 10 19:21:20.159579 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.159549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nh2g2" event={"ID":"c6f9c1d8-a78a-4541-bb95-e368edfee6fe","Type":"ContainerStarted","Data":"f55507fe3070df03c80608b38b654bbb871a172f2dfaab506a1d1e564365f4da"}
Sep 10 19:21:20.159736 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.159606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nh2g2" event={"ID":"c6f9c1d8-a78a-4541-bb95-e368edfee6fe","Type":"ContainerStarted","Data":"d99aa68c472b7e4e668c174d4c14b381ef871d1c5dd3b64c1b4cf5292607acd0"}
Sep 10 19:21:20.159736 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.159636    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:21:20.161189 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.161158    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6" event={"ID":"e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d","Type":"ContainerStarted","Data":"bd4ff207407cbd52b70919df4469194d51cc2257b821cddf0595f3d68346fc2b"}
Sep 10 19:21:20.162977 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.162953    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nhl6m" event={"ID":"e50d4c91-53ee-4496-9854-3a3abb0a1ca7","Type":"ContainerStarted","Data":"da1796079752b263ed1ce9df7bc497c11ed2d0af6fda8609f67c9508fa85a688"}
Sep 10 19:21:20.178695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.178640    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8dfpf" podStartSLOduration=66.115908476 podStartE2EDuration="1m11.1786246s" totalImagesPullingTime="5.062716124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:20.177380528 +0000 UTC m=+73.117414366" watchObservedRunningTime="2026-09-10 19:21:20.1786246 +0000 UTC m=+73.118658429"
Sep 10 19:21:20.193510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.193450    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nhl6m" podStartSLOduration=66.355904946 podStartE2EDuration="1m12.193429128s" totalImagesPullingTime="5.837524182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:20.192525803 +0000 UTC m=+73.132559638" watchObservedRunningTime="2026-09-10 19:21:20.193429128 +0000 UTC m=+73.133462962"
Sep 10 19:21:20.208683 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.208618    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-sk5q6" podStartSLOduration=65.997562078 podStartE2EDuration="1m11.208598272s" totalImagesPullingTime="5.211036194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:20.207335444 +0000 UTC m=+73.147369279" watchObservedRunningTime="2026-09-10 19:21:20.208598272 +0000 UTC m=+73.148632106"
Sep 10 19:21:20.226525 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.226401    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nh2g2" podStartSLOduration=34.114693106 podStartE2EDuration="39.226380383s" totalImagesPullingTime="5.111687277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:20.225794734 +0000 UTC m=+73.165828566" watchObservedRunningTime="2026-09-10 19:21:20.226380383 +0000 UTC m=+73.166414217"
Sep 10 19:21:20.303931 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.303862    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:20.369878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.369842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:20.885196 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:20.885160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:21.100468 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:21.100430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:21.201353 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:21.201312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:21.301759 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:21.301715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:21.396086 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:21.396051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:24.766745 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.766702    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fd968478d-wqvkt"]
Sep 10 19:21:24.804477 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.804432    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd968478d-wqvkt"]
Sep 10 19:21:24.804477 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.804471    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:24.807628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.807606    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 19:21:24.807778 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.807699    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 19:21:24.807928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.807914    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 19:21:24.808042 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.808024    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 19:21:24.808817 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.808801    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 19:21:24.808910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.808800    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 19:21:24.808910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.808870    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 19:21:24.809060 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.809041    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-c5tbm\""
Sep 10 19:21:24.994418 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.994378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-service-ca\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:24.994596 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.994428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-config\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:24.994596 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.994498    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-serving-cert\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:24.994596 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.994555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-oauth-serving-cert\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:24.994596 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.994577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-oauth-config\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:24.994747 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:24.994613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk4f7\" (UniqueName: \"kubernetes.io/projected/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-kube-api-access-sk4f7\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.095436 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.095347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-service-ca\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.095436 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.095388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-config\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.095436 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.095409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-serving-cert\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.095436 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.095436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-oauth-serving-cert\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.095702 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.095456    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-oauth-config\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.095702 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.095491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk4f7\" (UniqueName: \"kubernetes.io/projected/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-kube-api-access-sk4f7\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.096216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.096188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-oauth-serving-cert\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.096216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.096205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-service-ca\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.096366 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.096205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-config\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.097901 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.097861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-serving-cert\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.097986 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.097952    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-oauth-config\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.104257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.104231    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk4f7\" (UniqueName: \"kubernetes.io/projected/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-kube-api-access-sk4f7\") pod \"console-fd968478d-wqvkt\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") " pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.114508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.114479    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:25.118633 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.118597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:25.177459 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.177155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:25.406347 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:25.406308    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd968478d-wqvkt"]
Sep 10 19:21:25.410201 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:25.410174    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f0d2a5_d266_434b_b1d8_e6d748bb8a0c.slice/crio-d867104f0fae336d32f088fd10f1a6ae6d52132296564dcf7bd669ccf48103ec WatchSource:0}: Error finding container d867104f0fae336d32f088fd10f1a6ae6d52132296564dcf7bd669ccf48103ec: Status 404 returned error can't find the container with id d867104f0fae336d32f088fd10f1a6ae6d52132296564dcf7bd669ccf48103ec
Sep 10 19:21:26.159119 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.159061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:26.191264 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.191224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd968478d-wqvkt" event={"ID":"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c","Type":"ContainerStarted","Data":"d867104f0fae336d32f088fd10f1a6ae6d52132296564dcf7bd669ccf48103ec"}
Sep 10 19:21:26.195529 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.195497    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"]
Sep 10 19:21:26.212844 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.212806    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"]
Sep 10 19:21:26.213046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.212957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.215598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.215397    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 19:21:26.215598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.215426    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 19:21:26.215788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.215733    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gf7n4\""
Sep 10 19:21:26.216558 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.216531    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 19:21:26.232866 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.232832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:26.306999 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.306960    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svdcs\" (UniqueName: \"kubernetes.io/projected/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-kube-api-access-svdcs\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.307185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.307013    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.307185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.307103    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.307185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.307140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.407707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.407670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svdcs\" (UniqueName: \"kubernetes.io/projected/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-kube-api-access-svdcs\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.407928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.407718    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.407928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.407762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.407928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.407784    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.407928 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:26.407921    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 10 19:21:26.408149 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:26.408005    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls podName:85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:26.907984067 +0000 UTC m=+79.848017880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-ddmn2" (UID: "85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4") : secret "prometheus-operator-tls" not found
Sep 10 19:21:26.408542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.408496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.410740 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.410680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.416989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.416951    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svdcs\" (UniqueName: \"kubernetes.io/projected/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-kube-api-access-svdcs\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.911675 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:26.911633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:26.911843 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:26.911782    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 10 19:21:26.911917 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:26.911845    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls podName:85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:27.911826143 +0000 UTC m=+80.851859967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-ddmn2" (UID: "85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4") : secret "prometheus-operator-tls" not found
Sep 10 19:21:27.012326 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:27.012284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:27.918722 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:27.918675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:27.921574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:27.921543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ddmn2\" (UID: \"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:28.025718 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:28.025673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"
Sep 10 19:21:28.580518 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:28.580483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:28.618915 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:28.618866    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ddmn2"]
Sep 10 19:21:28.620922 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:28.620878    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ff17f3_fb2b_4cb1_a2ce_1ea3cd4fe9c4.slice/crio-a4bf9491bbd08e38cce40c9666709697ffc7cda2613d834df2997c3af2098456 WatchSource:0}: Error finding container a4bf9491bbd08e38cce40c9666709697ffc7cda2613d834df2997c3af2098456: Status 404 returned error can't find the container with id a4bf9491bbd08e38cce40c9666709697ffc7cda2613d834df2997c3af2098456
Sep 10 19:21:28.684740 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:28.684694    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:29.052824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:29.052780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:29.147247 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:29.147211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:29.204937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:29.204900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd968478d-wqvkt" event={"ID":"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c","Type":"ContainerStarted","Data":"28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89"}
Sep 10 19:21:29.206153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:29.206128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2" event={"ID":"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4","Type":"ContainerStarted","Data":"a4bf9491bbd08e38cce40c9666709697ffc7cda2613d834df2997c3af2098456"}
Sep 10 19:21:29.225177 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:29.225123    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fd968478d-wqvkt" podStartSLOduration=2.227392913 podStartE2EDuration="5.225106607s" totalImagesPullingTime="2.997713694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:29.223526144 +0000 UTC m=+82.163559981" watchObservedRunningTime="2026-09-10 19:21:29.225106607 +0000 UTC m=+82.165140439"
Sep 10 19:21:29.804017 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:29.803983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:29.904963 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:29.904823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:30.169729 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:30.169703    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nh2g2"
Sep 10 19:21:30.220581 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:30.220549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2" event={"ID":"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4","Type":"ContainerStarted","Data":"b365ed429cf239cfb580188f2f7127f286d5af4cce7066d79b22365f366f69e9"}
Sep 10 19:21:30.470209 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:30.470174    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:30.878488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:30.878402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:30.940627 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:30.940595    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:31.087206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:31.087169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:31.188388 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:31.188353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:31.226300 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:31.226263    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2" event={"ID":"85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4","Type":"ContainerStarted","Data":"cffca78e90d04f90acff8ff97ebc39a53a5e636de497d1760243c7a3150845bf"}
Sep 10 19:21:31.243057 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:31.243004    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-ddmn2" podStartSLOduration=3.735371727 podStartE2EDuration="5.242989677s" totalImagesPullingTime="1.50761795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:31.24280978 +0000 UTC m=+84.182843614" watchObservedRunningTime="2026-09-10 19:21:31.242989677 +0000 UTC m=+84.183023511"
Sep 10 19:21:31.244679 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:31.244649    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.309533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.309499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.406875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.406845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.528272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.528240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.547786 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.547750    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"]
Sep 10 19:21:33.551492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.551470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.553974 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.553946    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 19:21:33.554125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.554020    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tqlv9\""
Sep 10 19:21:33.554125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.554050    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 19:21:33.563735 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.563667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f98689ff-21ff-43fc-973a-325ae09b957c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.563735 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.563724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f98689ff-21ff-43fc-973a-325ae09b957c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.563898 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.563797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f98689ff-21ff-43fc-973a-325ae09b957c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.563898 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.563856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7jp6\" (UniqueName: \"kubernetes.io/projected/f98689ff-21ff-43fc-973a-325ae09b957c-kube-api-access-r7jp6\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.565308 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.565286    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"]
Sep 10 19:21:33.581922 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.581877    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wt4nz"]
Sep 10 19:21:33.585210 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.585188    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.587829 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.587807    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 19:21:33.588290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.588271    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 19:21:33.588397 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.588305    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dvs6s\""
Sep 10 19:21:33.588397 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.588377    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 19:21:33.621550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.621497    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.664257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f98689ff-21ff-43fc-973a-325ae09b957c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.664257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664263    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-accelerators-collector-config\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d96f0961-f8a6-460e-9f0a-22eb7c39b135-metrics-client-ca\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f98689ff-21ff-43fc-973a-325ae09b957c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.664516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7jp6\" (UniqueName: \"kubernetes.io/projected/f98689ff-21ff-43fc-973a-325ae09b957c-kube-api-access-r7jp6\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.664516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664473    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-wtmp\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664543    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-tls\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2gl2\" (UniqueName: \"kubernetes.io/projected/d96f0961-f8a6-460e-9f0a-22eb7c39b135-kube-api-access-g2gl2\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-sys\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-root\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-textfile\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.664760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.664733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f98689ff-21ff-43fc-973a-325ae09b957c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.665176 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.665154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f98689ff-21ff-43fc-973a-325ae09b957c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.666984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.666959    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f98689ff-21ff-43fc-973a-325ae09b957c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.666984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.666974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f98689ff-21ff-43fc-973a-325ae09b957c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.673826 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.673797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7jp6\" (UniqueName: \"kubernetes.io/projected/f98689ff-21ff-43fc-973a-325ae09b957c-kube-api-access-r7jp6\") pod \"openshift-state-metrics-657fd6c54-4grqx\" (UID: \"f98689ff-21ff-43fc-973a-325ae09b957c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.688112 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.688080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.765707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.765707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-wtmp\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-tls\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2gl2\" (UniqueName: \"kubernetes.io/projected/d96f0961-f8a6-460e-9f0a-22eb7c39b135-kube-api-access-g2gl2\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-sys\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-root\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-root\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765846    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-sys\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:33.765858    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765867    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-wtmp\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.765930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-textfile\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766002 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:33.765951    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-tls podName:d96f0961-f8a6-460e-9f0a-22eb7c39b135 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:34.265929096 +0000 UTC m=+87.205962934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-tls") pod "node-exporter-wt4nz" (UID: "d96f0961-f8a6-460e-9f0a-22eb7c39b135") : secret "node-exporter-tls" not found
Sep 10 19:21:33.766464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.766018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-accelerators-collector-config\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.766045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d96f0961-f8a6-460e-9f0a-22eb7c39b135-metrics-client-ca\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.766192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-textfile\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.766464    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-accelerators-collector-config\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.766556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.766520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d96f0961-f8a6-460e-9f0a-22eb7c39b135-metrics-client-ca\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.768150 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.768128    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.777823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.777791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2gl2\" (UniqueName: \"kubernetes.io/projected/d96f0961-f8a6-460e-9f0a-22eb7c39b135-kube-api-access-g2gl2\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:33.789484 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.789456    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.861102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.861018    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"
Sep 10 19:21:33.890153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.890115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:33.991265 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:33.991205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:34.014028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.013994    2562 patch_prober.go:28] interesting pod/image-registry-78875ff86d-blbk5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 19:21:34.014184 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.014049    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-78875ff86d-blbk5" podUID="e7f4b902-9333-4b0e-bd8f-3c01620d0e79" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:21:34.138126 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.137970    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx"]
Sep 10 19:21:34.140660 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:34.140636    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98689ff_21ff_43fc_973a_325ae09b957c.slice/crio-cfad9b1cfdf628609d67af98897ea064beedc7328803084ffa80e063e706e9ed WatchSource:0}: Error finding container cfad9b1cfdf628609d67af98897ea064beedc7328803084ffa80e063e706e9ed: Status 404 returned error can't find the container with id cfad9b1cfdf628609d67af98897ea064beedc7328803084ffa80e063e706e9ed
Sep 10 19:21:34.241367 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.241328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx" event={"ID":"f98689ff-21ff-43fc-973a-325ae09b957c","Type":"ContainerStarted","Data":"1487fbc59f744be6cbe9509fa18a9d72c0d0dfdcb2614ee3f5756e81ccfc1a72"}
Sep 10 19:21:34.241506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.241416    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx" event={"ID":"f98689ff-21ff-43fc-973a-325ae09b957c","Type":"ContainerStarted","Data":"cfad9b1cfdf628609d67af98897ea064beedc7328803084ffa80e063e706e9ed"}
Sep 10 19:21:34.269562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.269530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-tls\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:34.272016 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.271956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d96f0961-f8a6-460e-9f0a-22eb7c39b135-node-exporter-tls\") pod \"node-exporter-wt4nz\" (UID: \"d96f0961-f8a6-460e-9f0a-22eb7c39b135\") " pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:34.495896 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.495841    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wt4nz"
Sep 10 19:21:34.507617 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:34.507569    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96f0961_f8a6_460e_9f0a_22eb7c39b135.slice/crio-cc80c25392b7cd226eaae37a54ae46bab458b6256513d1470e7dd0d6af485f3e WatchSource:0}: Error finding container cc80c25392b7cd226eaae37a54ae46bab458b6256513d1470e7dd0d6af485f3e: Status 404 returned error can't find the container with id cc80c25392b7cd226eaae37a54ae46bab458b6256513d1470e7dd0d6af485f3e
Sep 10 19:21:34.531244 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.531208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:34.604549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:34.604518    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:35.115463 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:35.115422    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:35.115660 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:35.115494    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:35.122562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:35.121974    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:35.251175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:35.251080    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx" event={"ID":"f98689ff-21ff-43fc-973a-325ae09b957c","Type":"ContainerStarted","Data":"8108eba1c82500d09bdc46c377ff5dc546785cfc0281f794d0fb148292db9177"}
Sep 10 19:21:35.253110 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:35.253066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wt4nz" event={"ID":"d96f0961-f8a6-460e-9f0a-22eb7c39b135","Type":"ContainerStarted","Data":"cc80c25392b7cd226eaae37a54ae46bab458b6256513d1470e7dd0d6af485f3e"}
Sep 10 19:21:35.258141 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:35.258112    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:21:35.295345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:35.295282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:36.259430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:36.259390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx" event={"ID":"f98689ff-21ff-43fc-973a-325ae09b957c","Type":"ContainerStarted","Data":"85b6ac49502f1564f72c1cdf79cdd260c2ee298c74b191b81f239d5df257f636"}
Sep 10 19:21:36.261336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:36.261310    2562 generic.go:356] "Generic (PLEG): container finished" podID="d96f0961-f8a6-460e-9f0a-22eb7c39b135" containerID="70614f08a60b87c30295c073e8238bca23058f66025b9e1a8639050b4b62327a" exitCode=0
Sep 10 19:21:36.261490 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:36.261390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wt4nz" event={"ID":"d96f0961-f8a6-460e-9f0a-22eb7c39b135","Type":"ContainerDied","Data":"70614f08a60b87c30295c073e8238bca23058f66025b9e1a8639050b4b62327a"}
Sep 10 19:21:36.278720 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:36.278673    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4grqx" podStartSLOduration=1.908039064 podStartE2EDuration="3.278656952s" totalImagesPullingTime="1.370617888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:36.277376133 +0000 UTC m=+89.217409967" watchObservedRunningTime="2026-09-10 19:21:36.278656952 +0000 UTC m=+89.218690785"
Sep 10 19:21:36.513473 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:36.513382    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:36.616644 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:36.616595    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:36.797942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:36.797828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:37.147532 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:37.147504    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:21:37.267596 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:37.267554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wt4nz" event={"ID":"d96f0961-f8a6-460e-9f0a-22eb7c39b135","Type":"ContainerStarted","Data":"f65095652207af18f032731b539cf357c99a64a26ff001fe7173b82da2d0d44e"}
Sep 10 19:21:37.267985 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:37.267609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wt4nz" event={"ID":"d96f0961-f8a6-460e-9f0a-22eb7c39b135","Type":"ContainerStarted","Data":"335b6c662d9d4197096fb8d5cae6afb810979152cbf7b16c340fe377528b8911"}
Sep 10 19:21:37.293313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:37.293254    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wt4nz" podStartSLOduration=3.086687245 podStartE2EDuration="4.293236554s" totalImagesPullingTime="1.138419766s" totalInitContainerRuntime="68.129543ms" isStatefulPod=true podCreationTimestamp="2026-09-10 19:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:37.292095846 +0000 UTC m=+90.232129705" watchObservedRunningTime="2026-09-10 19:21:37.293236554 +0000 UTC m=+90.233270388"
Sep 10 19:21:37.324221 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:37.324180    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:37.398565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:37.398480    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:38.357433 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.357399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:38.376368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.376334    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"]
Sep 10 19:21:38.379819 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.379795    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:38.382512 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.382493    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 19:21:38.382812 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.382789    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2vtxx\""
Sep 10 19:21:38.388791 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.388765    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"]
Sep 10 19:21:38.406032 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.405997    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2437fe2e-7e58-4ee8-8247-e53eb4f69746-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-lkvxj\" (UID: \"2437fe2e-7e58-4ee8-8247-e53eb4f69746\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:38.406032 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.406034    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dkgz\" (UniqueName: \"kubernetes.io/projected/2437fe2e-7e58-4ee8-8247-e53eb4f69746-kube-api-access-7dkgz\") pod \"monitoring-plugin-85bbbbbf4-lkvxj\" (UID: \"2437fe2e-7e58-4ee8-8247-e53eb4f69746\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:38.507226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.507181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2437fe2e-7e58-4ee8-8247-e53eb4f69746-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-lkvxj\" (UID: \"2437fe2e-7e58-4ee8-8247-e53eb4f69746\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:38.507226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.507220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dkgz\" (UniqueName: \"kubernetes.io/projected/2437fe2e-7e58-4ee8-8247-e53eb4f69746-kube-api-access-7dkgz\") pod \"monitoring-plugin-85bbbbbf4-lkvxj\" (UID: \"2437fe2e-7e58-4ee8-8247-e53eb4f69746\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:38.507460 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:38.507340    2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 19:21:38.507460 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:21:38.507426    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2437fe2e-7e58-4ee8-8247-e53eb4f69746-monitoring-plugin-cert podName:2437fe2e-7e58-4ee8-8247-e53eb4f69746 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:39.007403806 +0000 UTC m=+91.947437634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2437fe2e-7e58-4ee8-8247-e53eb4f69746-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-lkvxj" (UID: "2437fe2e-7e58-4ee8-8247-e53eb4f69746") : secret "monitoring-plugin-cert" not found
Sep 10 19:21:38.518279 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.518242    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dkgz\" (UniqueName: \"kubernetes.io/projected/2437fe2e-7e58-4ee8-8247-e53eb4f69746-kube-api-access-7dkgz\") pod \"monitoring-plugin-85bbbbbf4-lkvxj\" (UID: \"2437fe2e-7e58-4ee8-8247-e53eb4f69746\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:38.710156 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.710123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:38.801543 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:38.801509    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:39.007306 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.007212    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd968478d-wqvkt"]
Sep 10 19:21:39.011877 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.011850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2437fe2e-7e58-4ee8-8247-e53eb4f69746-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-lkvxj\" (UID: \"2437fe2e-7e58-4ee8-8247-e53eb4f69746\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:39.014440 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.014415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2437fe2e-7e58-4ee8-8247-e53eb4f69746-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-lkvxj\" (UID: \"2437fe2e-7e58-4ee8-8247-e53eb4f69746\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:39.067012 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.066976    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:39.123506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.123474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:39.290663 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.290576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:39.568074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.567761    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"]
Sep 10 19:21:39.570731 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:39.570702    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2437fe2e_7e58_4ee8_8247_e53eb4f69746.slice/crio-926381efc60be9326c5e88f12c0b947ad6c8b39ea965f924c32f89e0d990985b WatchSource:0}: Error finding container 926381efc60be9326c5e88f12c0b947ad6c8b39ea965f924c32f89e0d990985b: Status 404 returned error can't find the container with id 926381efc60be9326c5e88f12c0b947ad6c8b39ea965f924c32f89e0d990985b
Sep 10 19:21:39.781539 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.781505    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:21:39.782246 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.782218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:39.786925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.786897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.789805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.789777    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 19:21:39.789805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.789795    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 19:21:39.790162 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790142    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 19:21:39.790213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790185    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 19:21:39.790256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790214    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 19:21:39.790584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790566    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fmme115atasud\""
Sep 10 19:21:39.790584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790575    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 19:21:39.790748 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790613    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 19:21:39.790748 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790643    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 19:21:39.790748 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790679    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 19:21:39.790925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790908    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 19:21:39.790925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790918    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 19:21:39.791040 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790924    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 19:21:39.791040 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.790937    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 19:21:39.791629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.791611    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bps68\""
Sep 10 19:21:39.791941 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.791923    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 19:21:39.800148 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.797510    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 19:21:39.811391 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.811257    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:21:39.819970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.819865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.819970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.819921    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.819970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.819951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-web-config\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.819970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.819969    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.819984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.819999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820161    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820197    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-config-out\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwdr6\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-kube-api-access-pwdr6\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-config\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.820694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.820548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-web-config\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921239    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921477    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-config-out\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwdr6\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-kube-api-access-pwdr6\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.921625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-config\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921641    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922377 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.922329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922469 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.922440    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.922716 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.922687    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.924187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.923338    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.924187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.923808    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.928671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.928189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.928671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.928303    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.928671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.921778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.928671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.928643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.929015 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.928968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.930033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.929368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.930033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.929541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.930033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.929546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-config\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.930033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.929764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-web-config\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.930033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.929991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.930672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.930608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.931278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.931070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.931278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.931239    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.931415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.931278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-config-out\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:39.936682 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:39.936662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwdr6\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-kube-api-access-pwdr6\") pod \"prometheus-k8s-0\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:40.102581 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:40.102491    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:40.285971 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:40.285929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj" event={"ID":"2437fe2e-7e58-4ee8-8247-e53eb4f69746","Type":"ContainerStarted","Data":"926381efc60be9326c5e88f12c0b947ad6c8b39ea965f924c32f89e0d990985b"}
Sep 10 19:21:40.403089 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:40.403054    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:21:40.404292 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:40.404267    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20bf616_beee_4c84_85e7_c87591a73631.slice/crio-08f41ba54eacc76987d3f1bd6c2a39d6f9fbcdba353e0e3d7c6c5787f6aa9093 WatchSource:0}: Error finding container 08f41ba54eacc76987d3f1bd6c2a39d6f9fbcdba353e0e3d7c6c5787f6aa9093: Status 404 returned error can't find the container with id 08f41ba54eacc76987d3f1bd6c2a39d6f9fbcdba353e0e3d7c6c5787f6aa9093
Sep 10 19:21:41.291984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.291939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj" event={"ID":"2437fe2e-7e58-4ee8-8247-e53eb4f69746","Type":"ContainerStarted","Data":"c18faaa5259bb4daddb703e8fbb0fadb193575df20211782699076a8cdd22f3a"}
Sep 10 19:21:41.292454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.292083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:41.293630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.293587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerStarted","Data":"08f41ba54eacc76987d3f1bd6c2a39d6f9fbcdba353e0e3d7c6c5787f6aa9093"}
Sep 10 19:21:41.297489 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.297463    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj"
Sep 10 19:21:41.310108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.310014    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lkvxj" podStartSLOduration=1.823948409 podStartE2EDuration="3.309996052s" totalImagesPullingTime="1.486047643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:41.308142314 +0000 UTC m=+94.248176149" watchObservedRunningTime="2026-09-10 19:21:41.309996052 +0000 UTC m=+94.250029938"
Sep 10 19:21:41.570532 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.570448    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fbf6cc6bd-fr7q6"]
Sep 10 19:21:41.574603 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.574574    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.582430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.582401    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 19:21:41.587835 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.587808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fbf6cc6bd-fr7q6"]
Sep 10 19:21:41.637105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.637070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gclg6\" (UniqueName: \"kubernetes.io/projected/16492380-5034-4fc3-ab45-fd5f9572a3b3-kube-api-access-gclg6\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.637290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.637112    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-oauth-serving-cert\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.637290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.637148    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-serving-cert\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.637290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.637175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-service-ca\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.637290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.637200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-trusted-ca-bundle\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.637290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.637226    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-config\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.637290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.637257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-oauth-config\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.737711 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.737674    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gclg6\" (UniqueName: \"kubernetes.io/projected/16492380-5034-4fc3-ab45-fd5f9572a3b3-kube-api-access-gclg6\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.737711 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.737716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-oauth-serving-cert\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.737964 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.737929    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-serving-cert\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.738024 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.737978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-service-ca\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.738024 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.738014    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-trusted-ca-bundle\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.738124 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.738043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-config\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.738805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.738445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-oauth-config\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.738805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.738610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-oauth-serving-cert\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.738805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.738695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-service-ca\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.738805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.738793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-config\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.739046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.738939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-trusted-ca-bundle\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.740820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.740798    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-oauth-config\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.740984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.740966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-serving-cert\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.746989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.746963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gclg6\" (UniqueName: \"kubernetes.io/projected/16492380-5034-4fc3-ab45-fd5f9572a3b3-kube-api-access-gclg6\") pod \"console-6fbf6cc6bd-fr7q6\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") " pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:41.886865 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:41.886770    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:42.161021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.160988    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fbf6cc6bd-fr7q6"]
Sep 10 19:21:42.164534 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:21:42.164505    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16492380_5034_4fc3_ab45_fd5f9572a3b3.slice/crio-5d5f505b638491b28fbc342515ac9b3cd896013f9fb2bc3efe8c43e3ebfbd387 WatchSource:0}: Error finding container 5d5f505b638491b28fbc342515ac9b3cd896013f9fb2bc3efe8c43e3ebfbd387: Status 404 returned error can't find the container with id 5d5f505b638491b28fbc342515ac9b3cd896013f9fb2bc3efe8c43e3ebfbd387
Sep 10 19:21:42.300085 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.300044    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bf616-beee-4c84-85e7-c87591a73631" containerID="1831212cd68c5fd279a2585eeb9e3468098891ffaadacfbd0b42cdc46daada5d" exitCode=0
Sep 10 19:21:42.300526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.300190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"1831212cd68c5fd279a2585eeb9e3468098891ffaadacfbd0b42cdc46daada5d"}
Sep 10 19:21:42.302087 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.302047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf6cc6bd-fr7q6" event={"ID":"16492380-5034-4fc3-ab45-fd5f9572a3b3","Type":"ContainerStarted","Data":"452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c"}
Sep 10 19:21:42.302142 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.302089    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf6cc6bd-fr7q6" event={"ID":"16492380-5034-4fc3-ab45-fd5f9572a3b3","Type":"ContainerStarted","Data":"5d5f505b638491b28fbc342515ac9b3cd896013f9fb2bc3efe8c43e3ebfbd387"}
Sep 10 19:21:42.357251 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.357193    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fbf6cc6bd-fr7q6" podStartSLOduration=1.35717593 podStartE2EDuration="1.35717593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:42.355809789 +0000 UTC m=+95.295843621" watchObservedRunningTime="2026-09-10 19:21:42.35717593 +0000 UTC m=+95.297209817"
Sep 10 19:21:42.388840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.388794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:42.458719 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:42.458638    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:43.005627 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:43.005588    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:43.105513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:43.105476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:44.174320 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:44.174285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:44.246101 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:44.246061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:45.044796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:45.044757    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78875ff86d-blbk5"]
Sep 10 19:21:45.292183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:45.292140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:45.393001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:45.392939    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:45.891528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:45.891494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:45.992065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:45.991977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:46.142582 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:46.142542    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xgn2n"
Sep 10 19:21:46.174559 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:46.174517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:46.274209 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:46.274128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:46.326971 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:46.326931    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerStarted","Data":"b8b23f2b6f00e2261c92de1c655f99147569a74c40ac33db86366f82f046ed24"}
Sep 10 19:21:46.327375 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:46.327005    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerStarted","Data":"fe17ce0b9996c96b4a9e22d218b1a10f39d12b22d7f1d69518e75139c5f6bf08"}
Sep 10 19:21:48.119683 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:48.119577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:48.221088 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:48.221050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:48.339670 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:48.339619    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerStarted","Data":"d3f93a8af4b5a938ef6be4af1badda834cf34e0d2c97b3b9428717ff11073963"}
Sep 10 19:21:48.339819 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:48.339717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerStarted","Data":"ee0d74cf63da7908deeee55027945e4319393f82d064146d63193807aaa68d95"}
Sep 10 19:21:48.339819 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:48.339758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerStarted","Data":"9783fe03bbb2fae2083ada925aaca94eaa0942299490dcd50b1c131bb2abe9b8"}
Sep 10 19:21:48.339819 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:48.339796    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerStarted","Data":"cf27d297a9ec6f61ada98dbee73261cfcef4e209a965d3af351694de86f99e62"}
Sep 10 19:21:48.375174 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:48.375045    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.000258661 podStartE2EDuration="9.375025868s" totalImagesPullingTime="6.355636846s" totalInitContainerRuntime="19.130361ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:48.373644165 +0000 UTC m=+101.313677997" watchObservedRunningTime="2026-09-10 19:21:48.375025868 +0000 UTC m=+101.315059703"
Sep 10 19:21:50.103558 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:50.103522    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:21:51.699452 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:51.699407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:51.764123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:51.764070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:51.886997 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:51.886963    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:51.887219 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:51.887034    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:51.892028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:51.892001    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:52.069318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:52.069229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:52.170852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:52.170815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:52.359783 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:52.359696    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:21:52.523958 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:52.523918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:52.625499 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:52.625399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:52.726508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:52.726470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:21:59.385672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:59.385635    2562 generic.go:356] "Generic (PLEG): container finished" podID="1a7b5479-66fb-4a0a-83d6-1a50656b07af" containerID="9a83b49a873a3d992dcfe1c11b1bb5f594e20eafe73b3df479fc046f02823459" exitCode=0
Sep 10 19:21:59.386065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:59.385706    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh" event={"ID":"1a7b5479-66fb-4a0a-83d6-1a50656b07af","Type":"ContainerDied","Data":"9a83b49a873a3d992dcfe1c11b1bb5f594e20eafe73b3df479fc046f02823459"}
Sep 10 19:21:59.386065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:21:59.386037    2562 scope.go:117] "RemoveContainer" containerID="9a83b49a873a3d992dcfe1c11b1bb5f594e20eafe73b3df479fc046f02823459"
Sep 10 19:22:00.391314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:00.391276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8szwh" event={"ID":"1a7b5479-66fb-4a0a-83d6-1a50656b07af","Type":"ContainerStarted","Data":"888ea28c7757a0a22f7e078fa15406ea0e8cb6bdce1cc2a850cc1e5b1c99629b"}
Sep 10 19:22:03.406180 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:03.406143    2562 generic.go:356] "Generic (PLEG): container finished" podID="20d869f4-b859-45f8-a21d-d90bf157087f" containerID="78d779a971c1635e1ed2c786796e29ed4029f86da3dd90a1168611dc282cd577" exitCode=0
Sep 10 19:22:03.406564 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:03.406194    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w" event={"ID":"20d869f4-b859-45f8-a21d-d90bf157087f","Type":"ContainerDied","Data":"78d779a971c1635e1ed2c786796e29ed4029f86da3dd90a1168611dc282cd577"}
Sep 10 19:22:03.406564 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:03.406554    2562 scope.go:117] "RemoveContainer" containerID="78d779a971c1635e1ed2c786796e29ed4029f86da3dd90a1168611dc282cd577"
Sep 10 19:22:04.028331 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.028292    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fd968478d-wqvkt" podUID="15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" containerName="console" containerID="cri-o://28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89" gracePeriod=15
Sep 10 19:22:04.303699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.303668    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:22:04.416181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.416147    2562 generic.go:356] "Generic (PLEG): container finished" podID="15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" containerID="28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89" exitCode=0
Sep 10 19:22:04.416621 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.416185    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd968478d-wqvkt" event={"ID":"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c","Type":"ContainerDied","Data":"28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89"}
Sep 10 19:22:04.416621 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.416210    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd968478d-wqvkt"
Sep 10 19:22:04.416621 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.416277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd968478d-wqvkt" event={"ID":"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c","Type":"ContainerDied","Data":"d867104f0fae336d32f088fd10f1a6ae6d52132296564dcf7bd669ccf48103ec"}
Sep 10 19:22:04.416621 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.416305    2562 scope.go:117] "RemoveContainer" containerID="28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89"
Sep 10 19:22:04.417796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.417773    2562 generic.go:356] "Generic (PLEG): container finished" podID="33fdd110-50e5-49e9-93f7-c57f7cfa5764" containerID="4a78fdd1a2ebf0dcf9e150d53657c330815f20b54266193dd0c6f07e0eb3b4a6" exitCode=0
Sep 10 19:22:04.417877 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.417825    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67" event={"ID":"33fdd110-50e5-49e9-93f7-c57f7cfa5764","Type":"ContainerDied","Data":"4a78fdd1a2ebf0dcf9e150d53657c330815f20b54266193dd0c6f07e0eb3b4a6"}
Sep 10 19:22:04.418195 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.418180    2562 scope.go:117] "RemoveContainer" containerID="4a78fdd1a2ebf0dcf9e150d53657c330815f20b54266193dd0c6f07e0eb3b4a6"
Sep 10 19:22:04.420047 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.420023    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-s4d5w" event={"ID":"20d869f4-b859-45f8-a21d-d90bf157087f","Type":"ContainerStarted","Data":"25a4d10161c98707e07d5746ec6a125b9362d5e21cccfeb9b2eeeb082757eb59"}
Sep 10 19:22:04.428752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.428731    2562 scope.go:117] "RemoveContainer" containerID="28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89"
Sep 10 19:22:04.429143 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:22:04.429120    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89\": container with ID starting with 28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89 not found: ID does not exist" containerID="28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89"
Sep 10 19:22:04.429225 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.429151    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89"} err="failed to get container status \"28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89\": rpc error: code = NotFound desc = could not find container \"28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89\": container with ID starting with 28cb6e7699bda0f94a73de682ed73636384f1255ef0f5af230a2dba00d4fdd89 not found: ID does not exist"
Sep 10 19:22:04.461964 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.461938    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-oauth-serving-cert\") pod \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") "
Sep 10 19:22:04.462127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.462010    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-config\") pod \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") "
Sep 10 19:22:04.462127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.462053    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk4f7\" (UniqueName: \"kubernetes.io/projected/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-kube-api-access-sk4f7\") pod \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") "
Sep 10 19:22:04.462127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.462110    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-service-ca\") pod \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") "
Sep 10 19:22:04.462286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.462162    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-oauth-config\") pod \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") "
Sep 10 19:22:04.462286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.462208    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-serving-cert\") pod \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\" (UID: \"15f0d2a5-d266-434b-b1d8-e6d748bb8a0c\") "
Sep 10 19:22:04.462585 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.462527    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" (UID: "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:04.463167 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.462776    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-config" (OuterVolumeSpecName: "console-config") pod "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" (UID: "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:04.463167 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.463135    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-service-ca" (OuterVolumeSpecName: "service-ca") pod "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" (UID: "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:04.465644 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.465615    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" (UID: "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:04.465765 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.465727    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" (UID: "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:04.465859 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.465825    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-kube-api-access-sk4f7" (OuterVolumeSpecName: "kube-api-access-sk4f7") pod "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" (UID: "15f0d2a5-d266-434b-b1d8-e6d748bb8a0c"). InnerVolumeSpecName "kube-api-access-sk4f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:04.563181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.563143    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk4f7\" (UniqueName: \"kubernetes.io/projected/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-kube-api-access-sk4f7\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:04.563181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.563180    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:04.563471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.563194    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-oauth-config\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:04.563471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.563207    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-serving-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:04.563471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.563219    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-oauth-serving-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:04.563471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.563231    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c-console-config\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:04.739160 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.739126    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd968478d-wqvkt"]
Sep 10 19:22:04.743759 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:04.743731    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fd968478d-wqvkt"]
Sep 10 19:22:05.427492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:05.427452    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-6rf67" event={"ID":"33fdd110-50e5-49e9-93f7-c57f7cfa5764","Type":"ContainerStarted","Data":"527fbf2870fe335da6ee845d5c3ed2730fb05f07728d829436b46c39eda4498d"}
Sep 10 19:22:05.694929 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:05.694831    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" path="/var/lib/kubelet/pods/15f0d2a5-d266-434b-b1d8-e6d748bb8a0c/volumes"
Sep 10 19:22:08.096225 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:08.096182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:10.069170 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.069125    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78875ff86d-blbk5" podUID="e7f4b902-9333-4b0e-bd8f-3c01620d0e79" containerName="registry" containerID="cri-o://ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d" gracePeriod=30
Sep 10 19:22:10.315249 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.315223    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:22:10.414465 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-trusted-ca\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlchs\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-kube-api-access-jlchs\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414545    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414589    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-image-registry-private-configuration\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414627    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-certificates\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414658    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-installation-pull-secrets\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414735    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-bound-sa-token\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414809    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-ca-trust-extracted\") pod \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\" (UID: \"e7f4b902-9333-4b0e-bd8f-3c01620d0e79\") "
Sep 10 19:22:10.414975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.414936    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:10.415205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.415183    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-trusted-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:10.415296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.415215    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:10.417394 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.417353    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:10.417514 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.417487    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:10.417576 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.417506    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:10.417576 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.417543    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:10.417651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.417624    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-kube-api-access-jlchs" (OuterVolumeSpecName: "kube-api-access-jlchs") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "kube-api-access-jlchs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:10.423946 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.423921    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e7f4b902-9333-4b0e-bd8f-3c01620d0e79" (UID: "e7f4b902-9333-4b0e-bd8f-3c01620d0e79"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:10.450973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.450934    2562 generic.go:356] "Generic (PLEG): container finished" podID="e7f4b902-9333-4b0e-bd8f-3c01620d0e79" containerID="ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d" exitCode=0
Sep 10 19:22:10.451125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.450984    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78875ff86d-blbk5"
Sep 10 19:22:10.451125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.450992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78875ff86d-blbk5" event={"ID":"e7f4b902-9333-4b0e-bd8f-3c01620d0e79","Type":"ContainerDied","Data":"ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d"}
Sep 10 19:22:10.451125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.451035    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78875ff86d-blbk5" event={"ID":"e7f4b902-9333-4b0e-bd8f-3c01620d0e79","Type":"ContainerDied","Data":"f18bfba335f81c1951470a41a98cbcea7a0406524da02d99bc60cd7734a25031"}
Sep 10 19:22:10.451125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.451055    2562 scope.go:117] "RemoveContainer" containerID="ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d"
Sep 10 19:22:10.460285 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.460264    2562 scope.go:117] "RemoveContainer" containerID="ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d"
Sep 10 19:22:10.460575 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:22:10.460557    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d\": container with ID starting with ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d not found: ID does not exist" containerID="ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d"
Sep 10 19:22:10.460621 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.460585    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d"} err="failed to get container status \"ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d\": rpc error: code = NotFound desc = could not find container \"ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d\": container with ID starting with ad70b20962241381a8ea80911c537aedb0d27bc0981d137c6469ede92ce00f4d not found: ID does not exist"
Sep 10 19:22:10.473584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.473552    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78875ff86d-blbk5"]
Sep 10 19:22:10.479158 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.479127    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78875ff86d-blbk5"]
Sep 10 19:22:10.516459 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.516427    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-ca-trust-extracted\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:10.516459 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.516455    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlchs\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-kube-api-access-jlchs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:10.516598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.516467    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-tls\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:10.516598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.516477    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-image-registry-private-configuration\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:10.516598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.516486    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-registry-certificates\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:10.516598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.516495    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-installation-pull-secrets\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:10.516598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:10.516503    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f4b902-9333-4b0e-bd8f-3c01620d0e79-bound-sa-token\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:11.695015 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:11.694978    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7f4b902-9333-4b0e-bd8f-3c01620d0e79" path="/var/lib/kubelet/pods/e7f4b902-9333-4b0e-bd8f-3c01620d0e79/volumes"
Sep 10 19:22:15.901391 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:15.901329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:15.961003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:15.960970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:27.827651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:27.827614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:31.375448 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:31.375414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:40.102991 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:40.102945    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:40.119206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:40.119169    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:40.596322 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:40.596289    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:53.048828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:53.048790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:53.224498 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:53.224464    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:22:58.461826 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.461787    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:22:58.462272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.462126    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="prometheus" containerID="cri-o://fe17ce0b9996c96b4a9e22d218b1a10f39d12b22d7f1d69518e75139c5f6bf08" gracePeriod=600
Sep 10 19:22:58.462272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.462165    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="thanos-sidecar" containerID="cri-o://cf27d297a9ec6f61ada98dbee73261cfcef4e209a965d3af351694de86f99e62" gracePeriod=600
Sep 10 19:22:58.462272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.462184    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d3f93a8af4b5a938ef6be4af1badda834cf34e0d2c97b3b9428717ff11073963" gracePeriod=600
Sep 10 19:22:58.462272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.462193    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-web" containerID="cri-o://9783fe03bbb2fae2083ada925aaca94eaa0942299490dcd50b1c131bb2abe9b8" gracePeriod=600
Sep 10 19:22:58.462272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.462149    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy" containerID="cri-o://ee0d74cf63da7908deeee55027945e4319393f82d064146d63193807aaa68d95" gracePeriod=600
Sep 10 19:22:58.462529 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.462349    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="config-reloader" containerID="cri-o://b8b23f2b6f00e2261c92de1c655f99147569a74c40ac33db86366f82f046ed24" gracePeriod=600
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660838    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bf616-beee-4c84-85e7-c87591a73631" containerID="d3f93a8af4b5a938ef6be4af1badda834cf34e0d2c97b3b9428717ff11073963" exitCode=0
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660870    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bf616-beee-4c84-85e7-c87591a73631" containerID="ee0d74cf63da7908deeee55027945e4319393f82d064146d63193807aaa68d95" exitCode=0
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660879    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bf616-beee-4c84-85e7-c87591a73631" containerID="9783fe03bbb2fae2083ada925aaca94eaa0942299490dcd50b1c131bb2abe9b8" exitCode=0
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"d3f93a8af4b5a938ef6be4af1badda834cf34e0d2c97b3b9428717ff11073963"}
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660913    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bf616-beee-4c84-85e7-c87591a73631" containerID="cf27d297a9ec6f61ada98dbee73261cfcef4e209a965d3af351694de86f99e62" exitCode=0
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660923    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bf616-beee-4c84-85e7-c87591a73631" containerID="b8b23f2b6f00e2261c92de1c655f99147569a74c40ac33db86366f82f046ed24" exitCode=0
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660930    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bf616-beee-4c84-85e7-c87591a73631" containerID="fe17ce0b9996c96b4a9e22d218b1a10f39d12b22d7f1d69518e75139c5f6bf08" exitCode=0
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.660983    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"ee0d74cf63da7908deeee55027945e4319393f82d064146d63193807aaa68d95"}
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.661015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"9783fe03bbb2fae2083ada925aaca94eaa0942299490dcd50b1c131bb2abe9b8"}
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.661042    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"cf27d297a9ec6f61ada98dbee73261cfcef4e209a965d3af351694de86f99e62"}
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.661073    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"b8b23f2b6f00e2261c92de1c655f99147569a74c40ac33db86366f82f046ed24"}
Sep 10 19:22:58.660878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.661097    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"fe17ce0b9996c96b4a9e22d218b1a10f39d12b22d7f1d69518e75139c5f6bf08"}
Sep 10 19:22:58.723381 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.723305    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:58.813021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.812655    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-web-config\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.813021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.812701    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-tls\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.813021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.812727    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-trusted-ca-bundle\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.813777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813463    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:58.813777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813624    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-serving-certs-ca-bundle\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.813777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813675    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-thanos-prometheus-http-client-file\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.813777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813710    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-db\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.813777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813736    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-kube-rbac-proxy\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.813777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813762    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-tls-assets\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813788    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-grpc-tls\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813816    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-config-out\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813852    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-0\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813903    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-1\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.813974    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-metrics-client-certs\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814009    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-config\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814063    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814096    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwdr6\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-kube-api-access-pwdr6\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814136    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-2\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814166    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-metrics-client-ca\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814198    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-kubelet-serving-ca-bundle\") pod \"e20bf616-beee-4c84-85e7-c87591a73631\" (UID: \"e20bf616-beee-4c84-85e7-c87591a73631\") "
Sep 10 19:22:58.814982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814472    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.814982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814488    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.814982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814795    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:58.814982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.814972    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:58.816500 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.816416    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:58.816952 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.816733    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:58.817358 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.817315    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:58.817605 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.817578    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:58.818357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.818306    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.818357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.818338    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.819533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.819498    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.820177 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.820143    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-config-out" (OuterVolumeSpecName: "config-out") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:58.820480 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.820421    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-kube-api-access-pwdr6" (OuterVolumeSpecName: "kube-api-access-pwdr6") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "kube-api-access-pwdr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:58.820798 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.820762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.821587 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.821553    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.821676 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.821623    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.821935 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.821917    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.822018 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.821927    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:58.822405 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.822387    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-config" (OuterVolumeSpecName: "config") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.836145 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.836102    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-web-config" (OuterVolumeSpecName: "web-config") pod "e20bf616-beee-4c84-85e7-c87591a73631" (UID: "e20bf616-beee-4c84-85e7-c87591a73631"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:58.915903 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915854    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-db\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915920    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-kube-rbac-proxy\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915935    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-tls-assets\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915947    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-grpc-tls\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915959    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e20bf616-beee-4c84-85e7-c87591a73631-config-out\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915971    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915983    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.915997    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916010    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-metrics-client-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916023    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916036    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-config\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916046    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwdr6\" (UniqueName: \"kubernetes.io/projected/e20bf616-beee-4c84-85e7-c87591a73631-kube-api-access-pwdr6\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916055    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916065    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-metrics-client-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916074    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e20bf616-beee-4c84-85e7-c87591a73631-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916085    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-web-config\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916098    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:58.916161 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:58.916111    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e20bf616-beee-4c84-85e7-c87591a73631-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:22:59.668202 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.668169    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.668202 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.668177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e20bf616-beee-4c84-85e7-c87591a73631","Type":"ContainerDied","Data":"08f41ba54eacc76987d3f1bd6c2a39d6f9fbcdba353e0e3d7c6c5787f6aa9093"}
Sep 10 19:22:59.668792 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.668272    2562 scope.go:117] "RemoveContainer" containerID="d3f93a8af4b5a938ef6be4af1badda834cf34e0d2c97b3b9428717ff11073963"
Sep 10 19:22:59.676419 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.676394    2562 scope.go:117] "RemoveContainer" containerID="ee0d74cf63da7908deeee55027945e4319393f82d064146d63193807aaa68d95"
Sep 10 19:22:59.684657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.684636    2562 scope.go:117] "RemoveContainer" containerID="9783fe03bbb2fae2083ada925aaca94eaa0942299490dcd50b1c131bb2abe9b8"
Sep 10 19:22:59.691825 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.691804    2562 scope.go:117] "RemoveContainer" containerID="cf27d297a9ec6f61ada98dbee73261cfcef4e209a965d3af351694de86f99e62"
Sep 10 19:22:59.700582 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.700550    2562 scope.go:117] "RemoveContainer" containerID="b8b23f2b6f00e2261c92de1c655f99147569a74c40ac33db86366f82f046ed24"
Sep 10 19:22:59.702729 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.702698    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:22:59.707151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.707123    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:22:59.711249 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.711223    2562 scope.go:117] "RemoveContainer" containerID="fe17ce0b9996c96b4a9e22d218b1a10f39d12b22d7f1d69518e75139c5f6bf08"
Sep 10 19:22:59.719260 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.719235    2562 scope.go:117] "RemoveContainer" containerID="1831212cd68c5fd279a2585eeb9e3468098891ffaadacfbd0b42cdc46daada5d"
Sep 10 19:22:59.744229 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744151    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:22:59.744488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744475    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7f4b902-9333-4b0e-bd8f-3c01620d0e79" containerName="registry"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744490    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7f4b902-9333-4b0e-bd8f-3c01620d0e79" containerName="registry"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744501    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="prometheus"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744506    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="prometheus"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744514    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-thanos"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744520    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-thanos"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744529    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="init-config-reloader"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744534    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="init-config-reloader"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744541    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-web"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744546    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-web"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744552    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="thanos-sidecar"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744557    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="thanos-sidecar"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744565    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="config-reloader"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744570    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="config-reloader"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744579    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" containerName="console"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744584    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" containerName="console"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744593    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744598    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744645    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="15f0d2a5-d266-434b-b1d8-e6d748bb8a0c" containerName="console"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744653    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy"
Sep 10 19:22:59.744648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744659    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="config-reloader"
Sep 10 19:22:59.745412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744666    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-thanos"
Sep 10 19:22:59.745412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744672    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7f4b902-9333-4b0e-bd8f-3c01620d0e79" containerName="registry"
Sep 10 19:22:59.745412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744678    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="prometheus"
Sep 10 19:22:59.745412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744683    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="thanos-sidecar"
Sep 10 19:22:59.745412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.744689    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bf616-beee-4c84-85e7-c87591a73631" containerName="kube-rbac-proxy-web"
Sep 10 19:22:59.749826 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.749806    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.766400 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766362    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 19:22:59.766400 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766393    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 19:22:59.766619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766396    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 19:22:59.766619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766396    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 19:22:59.766619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766554    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 19:22:59.766619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766604    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 19:22:59.766778 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766683    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 19:22:59.766778 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766686    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 19:22:59.766918 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.766876    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 19:22:59.767082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.767066    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 19:22:59.767152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.767080    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 19:22:59.767199 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.767165    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bps68\""
Sep 10 19:22:59.767325 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.767311    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 19:22:59.767443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.767424    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fmme115atasud\""
Sep 10 19:22:59.769437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.769415    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 19:22:59.797908 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.797866    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 19:22:59.800411 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.800382    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 19:22:59.800569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.800496    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:22:59.823488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823451    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823675 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823675 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823537    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823675 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823562    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-config\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823675 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823849    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.823875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823866    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.823978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a313236a-1122-4ffb-8d09-781be18c2a15-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824202 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq6z9\" (UniqueName: \"kubernetes.io/projected/a313236a-1122-4ffb-8d09-781be18c2a15-kube-api-access-tq6z9\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824202 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824083    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824202 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-web-config\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a313236a-1122-4ffb-8d09-781be18c2a15-config-out\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.824349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.824328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925009 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.924918    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-config\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925009 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.924971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925009 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925004    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925674 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925807 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a313236a-1122-4ffb-8d09-781be18c2a15-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.925873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq6z9\" (UniqueName: \"kubernetes.io/projected/a313236a-1122-4ffb-8d09-781be18c2a15-kube-api-access-tq6z9\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926007 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926007 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925922    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926007 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.925930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926166 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926010    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926166 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-web-config\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926166 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a313236a-1122-4ffb-8d09-781be18c2a15-config-out\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926166 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926112    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926166 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926146    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926429 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926429 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.926912 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.926803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.927802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.927470    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.927802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.927490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.929910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.928317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.929910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.928501    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.929910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.928675    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.929910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.928838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.929910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.928942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.929910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.929347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-config\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.929910 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.929459    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a313236a-1122-4ffb-8d09-781be18c2a15-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.930275 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.930173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.930443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.930419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.930571 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.930553    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a313236a-1122-4ffb-8d09-781be18c2a15-config-out\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.930979 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.930958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.931347 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.931328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.931421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.931401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a313236a-1122-4ffb-8d09-781be18c2a15-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.931731 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.931714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a313236a-1122-4ffb-8d09-781be18c2a15-web-config\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:22:59.936404 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:22:59.936380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq6z9\" (UniqueName: \"kubernetes.io/projected/a313236a-1122-4ffb-8d09-781be18c2a15-kube-api-access-tq6z9\") pod \"prometheus-k8s-0\" (UID: \"a313236a-1122-4ffb-8d09-781be18c2a15\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:23:00.058610 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:00.058518    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:23:00.353035 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:00.353009    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 19:23:00.354426 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:23:00.354397    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda313236a_1122_4ffb_8d09_781be18c2a15.slice/crio-161128b9388ad7f83231cf4943d7dac2897a2351f5ffc4b2e1d4056f38b9ba51 WatchSource:0}: Error finding container 161128b9388ad7f83231cf4943d7dac2897a2351f5ffc4b2e1d4056f38b9ba51: Status 404 returned error can't find the container with id 161128b9388ad7f83231cf4943d7dac2897a2351f5ffc4b2e1d4056f38b9ba51
Sep 10 19:23:00.673674 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:00.673638    2562 generic.go:356] "Generic (PLEG): container finished" podID="a313236a-1122-4ffb-8d09-781be18c2a15" containerID="04e2602cabd015e192daf71a659a0ad9405e09dae5135e439439eba95d1cc255" exitCode=0
Sep 10 19:23:00.674172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:00.673734    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerDied","Data":"04e2602cabd015e192daf71a659a0ad9405e09dae5135e439439eba95d1cc255"}
Sep 10 19:23:00.674172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:00.673811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerStarted","Data":"161128b9388ad7f83231cf4943d7dac2897a2351f5ffc4b2e1d4056f38b9ba51"}
Sep 10 19:23:01.683421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.683380    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerStarted","Data":"51d2979ee373901e43468ed38bd01c1dea5ba50bb44f0255e0aace7217c95883"}
Sep 10 19:23:01.683874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.683479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerStarted","Data":"9184e7691123357acee6bd49614a2fe333abecd6bfa9444ec871559e729b5ed7"}
Sep 10 19:23:01.683874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.683511    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerStarted","Data":"dcb76fd7331cd547f0eefbf48a09b6466e2af8aff966727fc7d43ae180a16dc8"}
Sep 10 19:23:01.683874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.683534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerStarted","Data":"d241abf42cf860e92b6a7809e6595cd742ce566927c17080d9806113e42306ab"}
Sep 10 19:23:01.683874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.683556    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerStarted","Data":"da4663fef92de655bd12c8ac47221fba9790b438e82c54c1b5878de0d85b44a6"}
Sep 10 19:23:01.683874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.683578    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a313236a-1122-4ffb-8d09-781be18c2a15","Type":"ContainerStarted","Data":"7cbacac4823f010a52a1e5401c4506a845a7044e3ce77330f70d3de191d3d16b"}
Sep 10 19:23:01.689043 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.689008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:01.694976 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.694950    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e20bf616-beee-4c84-85e7-c87591a73631" path="/var/lib/kubelet/pods/e20bf616-beee-4c84-85e7-c87591a73631/volumes"
Sep 10 19:23:01.726510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.726451    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.706892062 podStartE2EDuration="2.726432185s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.540123ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:01.724139545 +0000 UTC m=+174.664173379" watchObservedRunningTime="2026-09-10 19:23:01.726432185 +0000 UTC m=+174.666466018"
Sep 10 19:23:01.770458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:01.770418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:05.059036 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:05.058980    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:23:12.353814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:12.353776    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fbf6cc6bd-fr7q6"]
Sep 10 19:23:12.512003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:12.511964    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:12.612317 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:12.612232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:34.473109 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:34.473073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:34.590533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:34.590468    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:23:37.375866 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.375798    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fbf6cc6bd-fr7q6" podUID="16492380-5034-4fc3-ab45-fd5f9572a3b3" containerName="console" containerID="cri-o://452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c" gracePeriod=15
Sep 10 19:23:37.621550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.621522    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:23:37.642357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642317    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gclg6\" (UniqueName: \"kubernetes.io/projected/16492380-5034-4fc3-ab45-fd5f9572a3b3-kube-api-access-gclg6\") pod \"16492380-5034-4fc3-ab45-fd5f9572a3b3\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") "
Sep 10 19:23:37.642534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642396    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-service-ca\") pod \"16492380-5034-4fc3-ab45-fd5f9572a3b3\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") "
Sep 10 19:23:37.642534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642431    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-config\") pod \"16492380-5034-4fc3-ab45-fd5f9572a3b3\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") "
Sep 10 19:23:37.642534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-trusted-ca-bundle\") pod \"16492380-5034-4fc3-ab45-fd5f9572a3b3\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") "
Sep 10 19:23:37.642700 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642536    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-oauth-config\") pod \"16492380-5034-4fc3-ab45-fd5f9572a3b3\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") "
Sep 10 19:23:37.642700 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642565    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-oauth-serving-cert\") pod \"16492380-5034-4fc3-ab45-fd5f9572a3b3\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") "
Sep 10 19:23:37.642700 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642614    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-serving-cert\") pod \"16492380-5034-4fc3-ab45-fd5f9572a3b3\" (UID: \"16492380-5034-4fc3-ab45-fd5f9572a3b3\") "
Sep 10 19:23:37.643027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.642909    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-config" (OuterVolumeSpecName: "console-config") pod "16492380-5034-4fc3-ab45-fd5f9572a3b3" (UID: "16492380-5034-4fc3-ab45-fd5f9572a3b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:23:37.643027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.643012    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "16492380-5034-4fc3-ab45-fd5f9572a3b3" (UID: "16492380-5034-4fc3-ab45-fd5f9572a3b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:23:37.643173 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.643130    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "16492380-5034-4fc3-ab45-fd5f9572a3b3" (UID: "16492380-5034-4fc3-ab45-fd5f9572a3b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:23:37.643346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.643311    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "16492380-5034-4fc3-ab45-fd5f9572a3b3" (UID: "16492380-5034-4fc3-ab45-fd5f9572a3b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:23:37.645330 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.645295    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "16492380-5034-4fc3-ab45-fd5f9572a3b3" (UID: "16492380-5034-4fc3-ab45-fd5f9572a3b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:23:37.645461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.645364    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16492380-5034-4fc3-ab45-fd5f9572a3b3-kube-api-access-gclg6" (OuterVolumeSpecName: "kube-api-access-gclg6") pod "16492380-5034-4fc3-ab45-fd5f9572a3b3" (UID: "16492380-5034-4fc3-ab45-fd5f9572a3b3"). InnerVolumeSpecName "kube-api-access-gclg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:37.645461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.645382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "16492380-5034-4fc3-ab45-fd5f9572a3b3" (UID: "16492380-5034-4fc3-ab45-fd5f9572a3b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:23:37.743322 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.743280    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-trusted-ca-bundle\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:23:37.743322 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.743310    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-oauth-config\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:23:37.743322 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.743322    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-oauth-serving-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:23:37.743642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.743333    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-serving-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:23:37.743642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.743344    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gclg6\" (UniqueName: \"kubernetes.io/projected/16492380-5034-4fc3-ab45-fd5f9572a3b3-kube-api-access-gclg6\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:23:37.743642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.743353    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:23:37.743642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.743366    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16492380-5034-4fc3-ab45-fd5f9572a3b3-console-config\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:23:37.833951 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.833910    2562 generic.go:356] "Generic (PLEG): container finished" podID="16492380-5034-4fc3-ab45-fd5f9572a3b3" containerID="452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c" exitCode=0
Sep 10 19:23:37.834123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.833959    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbf6cc6bd-fr7q6"
Sep 10 19:23:37.834123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.833979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf6cc6bd-fr7q6" event={"ID":"16492380-5034-4fc3-ab45-fd5f9572a3b3","Type":"ContainerDied","Data":"452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c"}
Sep 10 19:23:37.834123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.834033    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbf6cc6bd-fr7q6" event={"ID":"16492380-5034-4fc3-ab45-fd5f9572a3b3","Type":"ContainerDied","Data":"5d5f505b638491b28fbc342515ac9b3cd896013f9fb2bc3efe8c43e3ebfbd387"}
Sep 10 19:23:37.834123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.834054    2562 scope.go:117] "RemoveContainer" containerID="452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c"
Sep 10 19:23:37.842404 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.842381    2562 scope.go:117] "RemoveContainer" containerID="452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c"
Sep 10 19:23:37.842736 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:23:37.842710    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c\": container with ID starting with 452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c not found: ID does not exist" containerID="452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c"
Sep 10 19:23:37.842822 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.842743    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c"} err="failed to get container status \"452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c\": rpc error: code = NotFound desc = could not find container \"452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c\": container with ID starting with 452d7518b9a5e24bafc686edba00cdcbe5a7e6a6c977d72662970cdae07a7f3c not found: ID does not exist"
Sep 10 19:23:37.853956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.853927    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fbf6cc6bd-fr7q6"]
Sep 10 19:23:37.862073 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:37.862045    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fbf6cc6bd-fr7q6"]
Sep 10 19:23:39.694187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:23:39.694153    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16492380-5034-4fc3-ab45-fd5f9572a3b3" path="/var/lib/kubelet/pods/16492380-5034-4fc3-ab45-fd5f9572a3b3/volumes"
Sep 10 19:24:00.059671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:00.059623    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:24:00.075704 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:00.075675    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:24:00.943292 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:00.943263    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 19:24:08.261828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:08.261795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:08.368821 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:08.368762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:09.912643 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:09.912606    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:10.013351 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:10.013317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:19.557276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:19.557231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:20.224812 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:20.224782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:20.347342 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:20.347310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:20.705915 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:20.705860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:20.815273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:20.815225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:20.928908 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:20.928849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:30.911483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:30.911392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:31.012299 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:31.012263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:54.630549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:54.630510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:24:54.730095 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:24:54.730054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:25:07.489484 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:25:07.489456    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:25:07.493969 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:25:07.493944    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:25:07.501086 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:25:07.501066    2562 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 19:25:07.629488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:25:07.629461    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:25:07.635078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:25:07.635051    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:26:39.279186 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:39.279146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:40.649973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:40.649934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:43.651362 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:43.651314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:43.752644 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:43.752610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:44.834611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:44.834570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:44.959372 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:44.959313    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:45.113566 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:45.113478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:45.496977 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:45.496941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:45.575290 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:45.575254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:47.715432 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:47.715394    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:47.913158 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:47.913119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:50.449459 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:50.449425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:50.556003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:50.555964    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:50.689522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:50.689488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:50.802139 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:50.802055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:56.165171 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:56.165136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:57.353378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:57.353345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:57.449902 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:57.449853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:59.520542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:59.520500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:26:59.616454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:26:59.616418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:01.263308 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:01.263220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:01.364469 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:01.364414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:01.422522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:01.422476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:03.916337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:03.916302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:04.020666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:04.020627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:05.694739 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:05.694695    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:12.578775 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:12.578731    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:12.677462 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:12.677423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:14.274288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:14.274250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:14.343881 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:14.343847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:20.171552 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.171509    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:20.226657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.226614    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"]
Sep 10 19:27:20.226982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.226968    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16492380-5034-4fc3-ab45-fd5f9572a3b3" containerName="console"
Sep 10 19:27:20.226982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.226983    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="16492380-5034-4fc3-ab45-fd5f9572a3b3" containerName="console"
Sep 10 19:27:20.227068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.227055    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="16492380-5034-4fc3-ab45-fd5f9572a3b3" containerName="console"
Sep 10 19:27:20.230110 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.230092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.233094 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.233060    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hws6j\""
Sep 10 19:27:20.234242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.234210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 19:27:20.234363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.234214    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 19:27:20.234363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.234221    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 19:27:20.234363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.234270    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 19:27:20.238361 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.238339    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:27:20.248433 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.248403    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"]
Sep 10 19:27:20.292690 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.292643    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30162097-2698-4b58-a980-b4f26d4e5793-cert\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.292863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.292753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30162097-2698-4b58-a980-b4f26d4e5793-manager-config\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.292863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.292820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdpfq\" (UniqueName: \"kubernetes.io/projected/30162097-2698-4b58-a980-b4f26d4e5793-kube-api-access-zdpfq\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.292863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.292852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30162097-2698-4b58-a980-b4f26d4e5793-metrics-cert\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.300680 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.300643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:20.394261 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.394215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30162097-2698-4b58-a980-b4f26d4e5793-manager-config\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.394441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.394276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdpfq\" (UniqueName: \"kubernetes.io/projected/30162097-2698-4b58-a980-b4f26d4e5793-kube-api-access-zdpfq\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.394441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.394297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30162097-2698-4b58-a980-b4f26d4e5793-metrics-cert\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.394441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.394321    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30162097-2698-4b58-a980-b4f26d4e5793-cert\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.395045 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.395022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30162097-2698-4b58-a980-b4f26d4e5793-manager-config\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.396828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.396805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30162097-2698-4b58-a980-b4f26d4e5793-cert\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.396925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.396882    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30162097-2698-4b58-a980-b4f26d4e5793-metrics-cert\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.442852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.442772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdpfq\" (UniqueName: \"kubernetes.io/projected/30162097-2698-4b58-a980-b4f26d4e5793-kube-api-access-zdpfq\") pod \"lws-controller-manager-57fccbbc8-gh4wn\" (UID: \"30162097-2698-4b58-a980-b4f26d4e5793\") " pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.539963 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.539918    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:20.821932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.821906    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"]
Sep 10 19:27:20.824369 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:27:20.824344    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30162097_2698_4b58_a980_b4f26d4e5793.slice/crio-1b1bacf40ec6e3d672d1fd8fcb9d30c17de9e76e178085b04e5f3f9d2c543553 WatchSource:0}: Error finding container 1b1bacf40ec6e3d672d1fd8fcb9d30c17de9e76e178085b04e5f3f9d2c543553: Status 404 returned error can't find the container with id 1b1bacf40ec6e3d672d1fd8fcb9d30c17de9e76e178085b04e5f3f9d2c543553
Sep 10 19:27:20.826305 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:20.826288    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:27:21.511068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:21.510987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:21.732953 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:21.732910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn" event={"ID":"30162097-2698-4b58-a980-b4f26d4e5793","Type":"ContainerStarted","Data":"1b1bacf40ec6e3d672d1fd8fcb9d30c17de9e76e178085b04e5f3f9d2c543553"}
Sep 10 19:27:22.115044 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.115006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:22.375087 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.374983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:22.556203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.556165    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"]
Sep 10 19:27:22.566682 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.566652    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.571985 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.571956    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 19:27:22.572206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.572062    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q6dtv\""
Sep 10 19:27:22.573040 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.573016    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 19:27:22.574344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.574323    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 19:27:22.574443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.574338    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 19:27:22.605691 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.605651    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"]
Sep 10 19:27:22.716004 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.715969    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-manifests\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.716004 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.716013    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-tmp\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.716214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.716079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.716214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.716103    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.716214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.716133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9x92\" (UniqueName: \"kubernetes.io/projected/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-kube-api-access-q9x92\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.816869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.816820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-manifests\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.816869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.816877    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-tmp\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.817125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.817067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.817183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.817137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.817240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.817197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9x92\" (UniqueName: \"kubernetes.io/projected/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-kube-api-access-q9x92\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.817399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.817285    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-manifests\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.819486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.819457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-tmp\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.819786 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.819764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.819841 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.819824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.834138 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.834105    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9x92\" (UniqueName: \"kubernetes.io/projected/5ecc8f01-fb7f-4b67-9006-19c705c8d59f-kube-api-access-q9x92\") pod \"opendatahub-operator-controller-manager-5ddcddd787-cxwdw\" (UID: \"5ecc8f01-fb7f-4b67-9006-19c705c8d59f\") " pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.842686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.842646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:22.880090 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.880054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:22.949071 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:22.949036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:23.045721 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.045647    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:23.146396 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.146355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:23.261486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.261444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:23.353148 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.353120    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"]
Sep 10 19:27:23.355146 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:27:23.355120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ecc8f01_fb7f_4b67_9006_19c705c8d59f.slice/crio-307c3871b52fd3801044de1cb0ee03cad3aa574f1a4e360a7f3665ab81b6342d WatchSource:0}: Error finding container 307c3871b52fd3801044de1cb0ee03cad3aa574f1a4e360a7f3665ab81b6342d: Status 404 returned error can't find the container with id 307c3871b52fd3801044de1cb0ee03cad3aa574f1a4e360a7f3665ab81b6342d
Sep 10 19:27:23.358238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.358194    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:23.742920 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.742851    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw" event={"ID":"5ecc8f01-fb7f-4b67-9006-19c705c8d59f","Type":"ContainerStarted","Data":"307c3871b52fd3801044de1cb0ee03cad3aa574f1a4e360a7f3665ab81b6342d"}
Sep 10 19:27:23.744358 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.744334    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn" event={"ID":"30162097-2698-4b58-a980-b4f26d4e5793","Type":"ContainerStarted","Data":"f98c48b1efe6bf43d4d215a942b62e6a84b7d065f21221a2b16a5c52a4292042"}
Sep 10 19:27:23.744499 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.744384    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:23.768506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:23.768450    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn" podStartSLOduration=1.465105175 podStartE2EDuration="3.768435118s" totalImagesPullingTime="2.303329943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:23.766877219 +0000 UTC m=+436.706911052" watchObservedRunningTime="2026-09-10 19:27:23.768435118 +0000 UTC m=+436.708468951"
Sep 10 19:27:26.760212 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:26.760171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw" event={"ID":"5ecc8f01-fb7f-4b67-9006-19c705c8d59f","Type":"ContainerStarted","Data":"8b3b090e0939b996ea425681e8241609b822294bf685ce8218682f9976738862"}
Sep 10 19:27:27.766236 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:27.766198    2562 generic.go:356] "Generic (PLEG): container finished" podID="5ecc8f01-fb7f-4b67-9006-19c705c8d59f" containerID="8b3b090e0939b996ea425681e8241609b822294bf685ce8218682f9976738862" exitCode=0
Sep 10 19:27:27.766678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:27.766281    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw" event={"ID":"5ecc8f01-fb7f-4b67-9006-19c705c8d59f","Type":"ContainerDied","Data":"8b3b090e0939b996ea425681e8241609b822294bf685ce8218682f9976738862"}
Sep 10 19:27:28.773518 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:28.773478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw" event={"ID":"5ecc8f01-fb7f-4b67-9006-19c705c8d59f","Type":"ContainerStarted","Data":"59adc0f9dc46e9049d23c37d2ff67d6df8381f01673ab40bca520fbe0ab044cd"}
Sep 10 19:27:28.774031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:28.773578    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:28.806784 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:28.806727    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw" podStartSLOduration=2.521462046 podStartE2EDuration="6.806709533s" totalImagesPullingTime="3.195466276s" totalInitContainerRuntime="1.089781211s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:28.805115241 +0000 UTC m=+441.745149073" watchObservedRunningTime="2026-09-10 19:27:28.806709533 +0000 UTC m=+441.746743365"
Sep 10 19:27:33.579299 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:33.579215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:33.730837 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:33.730798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:34.752349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:34.752316    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57fccbbc8-gh4wn"
Sep 10 19:27:39.780498 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:39.780461    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5ddcddd787-cxwdw"
Sep 10 19:27:41.997686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:41.997640    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:42.124361 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.124321    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:42.543791 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.543745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:42.875514 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.875432    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"]
Sep 10 19:27:42.987617 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.987580    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"]
Sep 10 19:27:42.987617 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.987614    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:42.997057 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.997029    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 19:27:42.997227 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.997029    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 19:27:42.997227 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.997066    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-flmcj\""
Sep 10 19:27:42.999790 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.999771    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvjw5\" (UniqueName: \"kubernetes.io/projected/e26c5c4f-a1e7-4762-ba4d-9066eae20633-kube-api-access-rvjw5\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.000056 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.999813    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tls-certs\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.000056 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:42.999863    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tmp\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.100515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.100476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvjw5\" (UniqueName: \"kubernetes.io/projected/e26c5c4f-a1e7-4762-ba4d-9066eae20633-kube-api-access-rvjw5\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.100698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.100530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tls-certs\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.100698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.100562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tmp\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.100698 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:27:43.100660    2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 19:27:43.100802 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:27:43.100753    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tls-certs podName:e26c5c4f-a1e7-4762-ba4d-9066eae20633 nodeName:}" failed. No retries permitted until 2026-09-10 19:27:43.600736067 +0000 UTC m=+456.540769878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tls-certs") pod "kube-auth-proxy-cf7f4d9c5-k6lcz" (UID: "e26c5c4f-a1e7-4762-ba4d-9066eae20633") : secret "kube-auth-proxy-tls" not found
Sep 10 19:27:43.102811 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.102787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tmp\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.117639 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.117607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvjw5\" (UniqueName: \"kubernetes.io/projected/e26c5c4f-a1e7-4762-ba4d-9066eae20633-kube-api-access-rvjw5\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.261769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.261733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:43.606124 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.606030    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tls-certs\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.608499 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.608466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e26c5c4f-a1e7-4762-ba4d-9066eae20633-tls-certs\") pod \"kube-auth-proxy-cf7f4d9c5-k6lcz\" (UID: \"e26c5c4f-a1e7-4762-ba4d-9066eae20633\") " pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:43.897386 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:43.897352    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"
Sep 10 19:27:44.178445 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:44.178342    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz"]
Sep 10 19:27:44.184882 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:27:44.184851    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26c5c4f_a1e7_4762_ba4d_9066eae20633.slice/crio-19c0321b38530f1651aed91dd90ca878ae6b1e0e045d300233f1e0dcfac53d38 WatchSource:0}: Error finding container 19c0321b38530f1651aed91dd90ca878ae6b1e0e045d300233f1e0dcfac53d38: Status 404 returned error can't find the container with id 19c0321b38530f1651aed91dd90ca878ae6b1e0e045d300233f1e0dcfac53d38
Sep 10 19:27:44.563515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:44.563434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:44.685390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:44.685337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:44.850737 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:44.850604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz" event={"ID":"e26c5c4f-a1e7-4762-ba4d-9066eae20633","Type":"ContainerStarted","Data":"19c0321b38530f1651aed91dd90ca878ae6b1e0e045d300233f1e0dcfac53d38"}
Sep 10 19:27:46.021049 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:46.021006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:47.867630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:47.867589    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz" event={"ID":"e26c5c4f-a1e7-4762-ba4d-9066eae20633","Type":"ContainerStarted","Data":"9cb1b60b67ae7c1f332186978cce5a8c5f3f1f1bad042652a55e2b929da384df"}
Sep 10 19:27:47.887449 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:47.887396    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-cf7f4d9c5-k6lcz" podStartSLOduration=3.173754119 podStartE2EDuration="5.887379339s" totalImagesPullingTime="2.71362522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:47.886551979 +0000 UTC m=+460.826585812" watchObservedRunningTime="2026-09-10 19:27:47.887379339 +0000 UTC m=+460.827413171"
Sep 10 19:27:49.781817 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:49.781780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:49.882912 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:49.882849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:50.772287 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:50.772246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:50.982245 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:50.982206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:51.263448 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:51.263412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:53.257229 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:53.257188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:53.378269 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:53.378237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:53.468560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:53.468524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:53.568581 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:53.568483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:53.971165 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:53.971129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:27:54.188650 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:27:54.188610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:05.901591 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:05.901554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:06.073519 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:06.073478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:06.242062 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:06.241977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:06.357028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:06.356993    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:07.191996 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:07.191958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:07.318634 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:07.318599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:07.503346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:07.503251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:54.208858 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:54.208812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:28:54.304726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:28:54.304690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:01.713725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:01.713637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:09.377538 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:09.377497    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:10.116394 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:10.116352    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:10.950784 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:10.950748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:13.872308 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:13.872274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:14.076544 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:14.076503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:14.357158 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:14.357123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:15.398337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:15.398303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:15.558464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:15.558430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:16.919537 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:16.919503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:17.592938 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:17.592871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:19.166764 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:19.166704    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:20.186609 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:20.186569    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:21.392767 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.392729    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"]
Sep 10 19:29:21.394634 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.394602    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:21.396123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.396106    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.399833 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.399805    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 19:29:21.401020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.401002    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-66xb2\""
Sep 10 19:29:21.401020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.401018    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 19:29:21.401187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.401006    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 19:29:21.401326 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.401312    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 19:29:21.408308 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.408284    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"]
Sep 10 19:29:21.560036 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.559996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f042119-a3cc-43f2-b6cf-080645b88822-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.560036 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.560045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k5cd\" (UniqueName: \"kubernetes.io/projected/6f042119-a3cc-43f2-b6cf-080645b88822-kube-api-access-6k5cd\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.560252 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.560074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6f042119-a3cc-43f2-b6cf-080645b88822-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.661409 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.661315    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k5cd\" (UniqueName: \"kubernetes.io/projected/6f042119-a3cc-43f2-b6cf-080645b88822-kube-api-access-6k5cd\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.661409 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.661359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6f042119-a3cc-43f2-b6cf-080645b88822-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.661658 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.661565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f042119-a3cc-43f2-b6cf-080645b88822-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.662140 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.662116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6f042119-a3cc-43f2-b6cf-080645b88822-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.664086 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.664068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f042119-a3cc-43f2-b6cf-080645b88822-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.685553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.685524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k5cd\" (UniqueName: \"kubernetes.io/projected/6f042119-a3cc-43f2-b6cf-080645b88822-kube-api-access-6k5cd\") pod \"kuadrant-console-plugin-fbdd4f445-g9khm\" (UID: \"6f042119-a3cc-43f2-b6cf-080645b88822\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.715209 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.715169    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"
Sep 10 19:29:21.739226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:21.739179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:22.010039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:22.010015    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm"]
Sep 10 19:29:22.012180 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:29:22.012149    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f042119_a3cc_43f2_b6cf_080645b88822.slice/crio-4b530e88ee257d736786f259986271db9ae3c4bc8c27a7ce6c34af7f59221dd1 WatchSource:0}: Error finding container 4b530e88ee257d736786f259986271db9ae3c4bc8c27a7ce6c34af7f59221dd1: Status 404 returned error can't find the container with id 4b530e88ee257d736786f259986271db9ae3c4bc8c27a7ce6c34af7f59221dd1
Sep 10 19:29:22.293924 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:22.293822    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm" event={"ID":"6f042119-a3cc-43f2-b6cf-080645b88822","Type":"ContainerStarted","Data":"4b530e88ee257d736786f259986271db9ae3c4bc8c27a7ce6c34af7f59221dd1"}
Sep 10 19:29:24.321257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:24.321214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:24.419909 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:24.419850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:24.715705 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:24.715651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:24.815993 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:24.815953    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:25.113736 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:25.113630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:25.276448 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:25.276411    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:27.330262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:27.330221    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm" event={"ID":"6f042119-a3cc-43f2-b6cf-080645b88822","Type":"ContainerStarted","Data":"c49f8312a32e7f64f3f5fd0cd1d9d8bb393585c1ac9bd6cf21787cad99b18832"}
Sep 10 19:29:27.352535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:27.352480    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-g9khm" podStartSLOduration=1.7423827429999998 podStartE2EDuration="6.352464686s" totalImagesPullingTime="4.610081943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:27.351093507 +0000 UTC m=+560.291127341" watchObservedRunningTime="2026-09-10 19:29:27.352464686 +0000 UTC m=+560.292498518"
Sep 10 19:29:33.160268 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:33.160222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:33.258826 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:33.258784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:36.157030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:36.156988    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:36.255850 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:36.255813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:38.601478 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:38.601436    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:38.705554 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:38.705511    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:45.752074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:45.752018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:45.859207 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:45.859172    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:45.953278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:45.953240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:46.052707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:46.052591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:46.153407 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:46.153367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:46.235744 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:46.235700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:54.928876 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:54.928836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:54.992686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:54.992642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:55.129134 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:55.129095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:55.202870 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:55.202771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:57.733699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:57.733665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:29:57.837209 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:29:57.837175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.300431 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:00.300391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:00.403063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:00.403012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:02.681260 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:02.681217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:03.369264 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:03.369225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:05.024560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.024517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:05.138104 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.138073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:05.242847 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.242808    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:05.336996 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.336912    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:30:05.340282 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.340263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.343136 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.343109    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 19:30:05.349357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.349327    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:30:05.381994 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.381960    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:30:05.450403 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.450360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb2sn\" (UniqueName: \"kubernetes.io/projected/395914f3-96fc-4949-9fc1-e95f364e37a7-kube-api-access-vb2sn\") pod \"limitador-limitador-84994f7ffc-5b7nw\" (UID: \"395914f3-96fc-4949-9fc1-e95f364e37a7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.450589 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.450513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/395914f3-96fc-4949-9fc1-e95f364e37a7-config-file\") pod \"limitador-limitador-84994f7ffc-5b7nw\" (UID: \"395914f3-96fc-4949-9fc1-e95f364e37a7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.551463 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.551422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/395914f3-96fc-4949-9fc1-e95f364e37a7-config-file\") pod \"limitador-limitador-84994f7ffc-5b7nw\" (UID: \"395914f3-96fc-4949-9fc1-e95f364e37a7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.551654 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.551492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb2sn\" (UniqueName: \"kubernetes.io/projected/395914f3-96fc-4949-9fc1-e95f364e37a7-kube-api-access-vb2sn\") pod \"limitador-limitador-84994f7ffc-5b7nw\" (UID: \"395914f3-96fc-4949-9fc1-e95f364e37a7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.552148 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.552127    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/395914f3-96fc-4949-9fc1-e95f364e37a7-config-file\") pod \"limitador-limitador-84994f7ffc-5b7nw\" (UID: \"395914f3-96fc-4949-9fc1-e95f364e37a7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.560298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.560273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb2sn\" (UniqueName: \"kubernetes.io/projected/395914f3-96fc-4949-9fc1-e95f364e37a7-kube-api-access-vb2sn\") pod \"limitador-limitador-84994f7ffc-5b7nw\" (UID: \"395914f3-96fc-4949-9fc1-e95f364e37a7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.651507 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.651466    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:05.792868 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.792832    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pgtvp"]
Sep 10 19:30:05.796415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.796390    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp"
Sep 10 19:30:05.800021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.799993    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sxljm\""
Sep 10 19:30:05.803352 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.803327    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pgtvp"]
Sep 10 19:30:05.836068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.836035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:05.929982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.929950    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:30:05.932439 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:30:05.932407    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395914f3_96fc_4949_9fc1_e95f364e37a7.slice/crio-0aac6e86ebd879582b4bf664684216eee8f389aee110d0fa69d971a4d2db1313 WatchSource:0}: Error finding container 0aac6e86ebd879582b4bf664684216eee8f389aee110d0fa69d971a4d2db1313: Status 404 returned error can't find the container with id 0aac6e86ebd879582b4bf664684216eee8f389aee110d0fa69d971a4d2db1313
Sep 10 19:30:05.953959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.953922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sj8s\" (UniqueName: \"kubernetes.io/projected/1d49ba9f-e52e-4dbd-9412-5bef7d17664c-kube-api-access-9sj8s\") pod \"authorino-7bcfd8f9cd-pgtvp\" (UID: \"1d49ba9f-e52e-4dbd-9412-5bef7d17664c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp"
Sep 10 19:30:05.969033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.968993    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-5spv7"]
Sep 10 19:30:05.972453 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.972434    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5spv7"
Sep 10 19:30:05.981863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:05.981837    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5spv7"]
Sep 10 19:30:06.054793 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.054757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdjg\" (UniqueName: \"kubernetes.io/projected/1cf860dc-a732-49b9-a604-23f5c0dbb509-kube-api-access-jpdjg\") pod \"authorino-59c974bd48-5spv7\" (UID: \"1cf860dc-a732-49b9-a604-23f5c0dbb509\") " pod="kuadrant-system/authorino-59c974bd48-5spv7"
Sep 10 19:30:06.055239 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.054853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sj8s\" (UniqueName: \"kubernetes.io/projected/1d49ba9f-e52e-4dbd-9412-5bef7d17664c-kube-api-access-9sj8s\") pod \"authorino-7bcfd8f9cd-pgtvp\" (UID: \"1d49ba9f-e52e-4dbd-9412-5bef7d17664c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp"
Sep 10 19:30:06.070528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.070494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sj8s\" (UniqueName: \"kubernetes.io/projected/1d49ba9f-e52e-4dbd-9412-5bef7d17664c-kube-api-access-9sj8s\") pod \"authorino-7bcfd8f9cd-pgtvp\" (UID: \"1d49ba9f-e52e-4dbd-9412-5bef7d17664c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp"
Sep 10 19:30:06.107366 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.107310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp"
Sep 10 19:30:06.156478 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.156423    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdjg\" (UniqueName: \"kubernetes.io/projected/1cf860dc-a732-49b9-a604-23f5c0dbb509-kube-api-access-jpdjg\") pod \"authorino-59c974bd48-5spv7\" (UID: \"1cf860dc-a732-49b9-a604-23f5c0dbb509\") " pod="kuadrant-system/authorino-59c974bd48-5spv7"
Sep 10 19:30:06.169640 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.169608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdjg\" (UniqueName: \"kubernetes.io/projected/1cf860dc-a732-49b9-a604-23f5c0dbb509-kube-api-access-jpdjg\") pod \"authorino-59c974bd48-5spv7\" (UID: \"1cf860dc-a732-49b9-a604-23f5c0dbb509\") " pod="kuadrant-system/authorino-59c974bd48-5spv7"
Sep 10 19:30:06.282956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.282861    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5spv7"
Sep 10 19:30:06.341704 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.341671    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:06.381935 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.381852    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pgtvp"]
Sep 10 19:30:06.384135 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:30:06.384098    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d49ba9f_e52e_4dbd_9412_5bef7d17664c.slice/crio-5a8126aae28a577116de97e0f6e8a714cdfb8b06aa7451fb126dbdcf9cfa1161 WatchSource:0}: Error finding container 5a8126aae28a577116de97e0f6e8a714cdfb8b06aa7451fb126dbdcf9cfa1161: Status 404 returned error can't find the container with id 5a8126aae28a577116de97e0f6e8a714cdfb8b06aa7451fb126dbdcf9cfa1161
Sep 10 19:30:06.508489 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.508449    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp" event={"ID":"1d49ba9f-e52e-4dbd-9412-5bef7d17664c","Type":"ContainerStarted","Data":"5a8126aae28a577116de97e0f6e8a714cdfb8b06aa7451fb126dbdcf9cfa1161"}
Sep 10 19:30:06.509642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.509612    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw" event={"ID":"395914f3-96fc-4949-9fc1-e95f364e37a7","Type":"ContainerStarted","Data":"0aac6e86ebd879582b4bf664684216eee8f389aee110d0fa69d971a4d2db1313"}
Sep 10 19:30:06.556694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:06.556658    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5spv7"]
Sep 10 19:30:06.559108 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:30:06.559069    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cf860dc_a732_49b9_a604_23f5c0dbb509.slice/crio-19712439c1e530588b78850e4ed7594c9bd12fa8e65363ae7ff543b85c8b46ae WatchSource:0}: Error finding container 19712439c1e530588b78850e4ed7594c9bd12fa8e65363ae7ff543b85c8b46ae: Status 404 returned error can't find the container with id 19712439c1e530588b78850e4ed7594c9bd12fa8e65363ae7ff543b85c8b46ae
Sep 10 19:30:07.519707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:07.519503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5spv7" event={"ID":"1cf860dc-a732-49b9-a604-23f5c0dbb509","Type":"ContainerStarted","Data":"19712439c1e530588b78850e4ed7594c9bd12fa8e65363ae7ff543b85c8b46ae"}
Sep 10 19:30:07.535831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:07.535683    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:30:07.545948 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:07.545916    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:30:07.689946 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:07.689913    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:30:07.702326 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:07.702295    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:30:10.534699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:10.534662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5spv7" event={"ID":"1cf860dc-a732-49b9-a604-23f5c0dbb509","Type":"ContainerStarted","Data":"8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d"}
Sep 10 19:30:10.536296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:10.536270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp" event={"ID":"1d49ba9f-e52e-4dbd-9412-5bef7d17664c","Type":"ContainerStarted","Data":"209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261"}
Sep 10 19:30:10.552695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:10.552620    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-5spv7" podStartSLOduration=2.6346447189999997 podStartE2EDuration="5.552600997s" totalImagesPullingTime="2.917956278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:10.550309746 +0000 UTC m=+603.490343579" watchObservedRunningTime="2026-09-10 19:30:10.552600997 +0000 UTC m=+603.492634830"
Sep 10 19:30:10.572203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:10.572146    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp" podStartSLOduration=2.490160243 podStartE2EDuration="5.572126446s" totalImagesPullingTime="3.081966203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:10.570750979 +0000 UTC m=+603.510784812" watchObservedRunningTime="2026-09-10 19:30:10.572126446 +0000 UTC m=+603.512160279"
Sep 10 19:30:10.585921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:10.585864    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pgtvp"]
Sep 10 19:30:12.545359 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:12.545294    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp" podUID="1d49ba9f-e52e-4dbd-9412-5bef7d17664c" containerName="authorino" containerID="cri-o://209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261" gracePeriod=30
Sep 10 19:30:12.792879 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:12.792849    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp"
Sep 10 19:30:12.916203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:12.916170    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sj8s\" (UniqueName: \"kubernetes.io/projected/1d49ba9f-e52e-4dbd-9412-5bef7d17664c-kube-api-access-9sj8s\") pod \"1d49ba9f-e52e-4dbd-9412-5bef7d17664c\" (UID: \"1d49ba9f-e52e-4dbd-9412-5bef7d17664c\") "
Sep 10 19:30:12.918480 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:12.918450    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d49ba9f-e52e-4dbd-9412-5bef7d17664c-kube-api-access-9sj8s" (OuterVolumeSpecName: "kube-api-access-9sj8s") pod "1d49ba9f-e52e-4dbd-9412-5bef7d17664c" (UID: "1d49ba9f-e52e-4dbd-9412-5bef7d17664c"). InnerVolumeSpecName "kube-api-access-9sj8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:13.016919 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.016846    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sj8s\" (UniqueName: \"kubernetes.io/projected/1d49ba9f-e52e-4dbd-9412-5bef7d17664c-kube-api-access-9sj8s\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:30:13.551169 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.551128    2562 generic.go:356] "Generic (PLEG): container finished" podID="1d49ba9f-e52e-4dbd-9412-5bef7d17664c" containerID="209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261" exitCode=0
Sep 10 19:30:13.551542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.551186    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp"
Sep 10 19:30:13.551542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.551219    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp" event={"ID":"1d49ba9f-e52e-4dbd-9412-5bef7d17664c","Type":"ContainerDied","Data":"209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261"}
Sep 10 19:30:13.551542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.551267    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pgtvp" event={"ID":"1d49ba9f-e52e-4dbd-9412-5bef7d17664c","Type":"ContainerDied","Data":"5a8126aae28a577116de97e0f6e8a714cdfb8b06aa7451fb126dbdcf9cfa1161"}
Sep 10 19:30:13.551542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.551285    2562 scope.go:117] "RemoveContainer" containerID="209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261"
Sep 10 19:30:13.560092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.560068    2562 scope.go:117] "RemoveContainer" containerID="209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261"
Sep 10 19:30:13.560393 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:30:13.560375    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261\": container with ID starting with 209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261 not found: ID does not exist" containerID="209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261"
Sep 10 19:30:13.560461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.560402    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261"} err="failed to get container status \"209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261\": rpc error: code = NotFound desc = could not find container \"209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261\": container with ID starting with 209cee29da75738d13319eed0c76003cd7abd5add302fe746f19b5cc584fe261 not found: ID does not exist"
Sep 10 19:30:13.574650 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.574621    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pgtvp"]
Sep 10 19:30:13.579401 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.579360    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pgtvp"]
Sep 10 19:30:13.697678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:13.697640    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d49ba9f-e52e-4dbd-9412-5bef7d17664c" path="/var/lib/kubelet/pods/1d49ba9f-e52e-4dbd-9412-5bef7d17664c/volumes"
Sep 10 19:30:15.851802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:15.851776    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:15.952350 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:15.952309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:16.568197 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:16.568152    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw" event={"ID":"395914f3-96fc-4949-9fc1-e95f364e37a7","Type":"ContainerStarted","Data":"2b2c70406613b2834a78e68c4eea02254bf7d150bd04c51cd028c0cba4970cf6"}
Sep 10 19:30:16.568378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:16.568274    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:16.587500 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:16.587444    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw" podStartSLOduration=1.7299452739999999 podStartE2EDuration="11.587425112s" totalImagesPullingTime="9.857479838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:16.586181267 +0000 UTC m=+609.526215100" watchObservedRunningTime="2026-09-10 19:30:16.587425112 +0000 UTC m=+609.527458941"
Sep 10 19:30:27.574183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:27.574150    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"
Sep 10 19:30:31.396831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:31.396736    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:31.508365 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:31.508329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:31.883283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:31.883185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:43.861079 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:43.861037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:44.090918 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.090872    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-5m69n"]
Sep 10 19:30:44.091267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.091253    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d49ba9f-e52e-4dbd-9412-5bef7d17664c" containerName="authorino"
Sep 10 19:30:44.091313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.091270    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d49ba9f-e52e-4dbd-9412-5bef7d17664c" containerName="authorino"
Sep 10 19:30:44.091357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.091347    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d49ba9f-e52e-4dbd-9412-5bef7d17664c" containerName="authorino"
Sep 10 19:30:44.097119 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.097092    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5m69n"
Sep 10 19:30:44.101787 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.101756    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-5m69n"]
Sep 10 19:30:44.167903 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.167851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:44.197154 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.197116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqbvf\" (UniqueName: \"kubernetes.io/projected/73756f49-468e-48d3-a1d8-3c3f87e39833-kube-api-access-zqbvf\") pod \"authorino-557f876f69-5m69n\" (UID: \"73756f49-468e-48d3-a1d8-3c3f87e39833\") " pod="kuadrant-system/authorino-557f876f69-5m69n"
Sep 10 19:30:44.220377 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.220338    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-5m69n"]
Sep 10 19:30:44.220635 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:30:44.220613    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zqbvf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-5m69n" podUID="73756f49-468e-48d3-a1d8-3c3f87e39833"
Sep 10 19:30:44.248166 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.248132    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-xfg6z"]
Sep 10 19:30:44.251325 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.251303    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.254110 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.254086    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 19:30:44.258579 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.258547    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-xfg6z"]
Sep 10 19:30:44.297900 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.297844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvf\" (UniqueName: \"kubernetes.io/projected/73756f49-468e-48d3-a1d8-3c3f87e39833-kube-api-access-zqbvf\") pod \"authorino-557f876f69-5m69n\" (UID: \"73756f49-468e-48d3-a1d8-3c3f87e39833\") " pod="kuadrant-system/authorino-557f876f69-5m69n"
Sep 10 19:30:44.311678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.311644    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqbvf\" (UniqueName: \"kubernetes.io/projected/73756f49-468e-48d3-a1d8-3c3f87e39833-kube-api-access-zqbvf\") pod \"authorino-557f876f69-5m69n\" (UID: \"73756f49-468e-48d3-a1d8-3c3f87e39833\") " pod="kuadrant-system/authorino-557f876f69-5m69n"
Sep 10 19:30:44.323767 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.323734    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-xfg6z"]
Sep 10 19:30:44.324056 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:30:44.324035    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-n5nvb tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-n5nvb tls-cert]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-xfg6z" podUID="ed7719e2-3efb-4ab7-8912-965c94a54130"
Sep 10 19:30:44.355221 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.355180    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69c4d5d9c9-lrzzp"]
Sep 10 19:30:44.358409 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.358383    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.376781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.376748    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69c4d5d9c9-lrzzp"]
Sep 10 19:30:44.398315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.398268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed7719e2-3efb-4ab7-8912-965c94a54130-tls-cert\") pod \"authorino-6d4dfc649f-xfg6z\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") " pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.398489 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.398333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5nvb\" (UniqueName: \"kubernetes.io/projected/ed7719e2-3efb-4ab7-8912-965c94a54130-kube-api-access-n5nvb\") pod \"authorino-6d4dfc649f-xfg6z\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") " pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.499121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.499028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed7719e2-3efb-4ab7-8912-965c94a54130-tls-cert\") pod \"authorino-6d4dfc649f-xfg6z\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") " pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.499121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.499078    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bc2d132-7270-4660-bffc-ab06e9f3b0af-tls-cert\") pod \"authorino-69c4d5d9c9-lrzzp\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") " pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.499121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.499105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5nvb\" (UniqueName: \"kubernetes.io/projected/ed7719e2-3efb-4ab7-8912-965c94a54130-kube-api-access-n5nvb\") pod \"authorino-6d4dfc649f-xfg6z\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") " pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.499344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.499151    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7t8t\" (UniqueName: \"kubernetes.io/projected/9bc2d132-7270-4660-bffc-ab06e9f3b0af-kube-api-access-j7t8t\") pod \"authorino-69c4d5d9c9-lrzzp\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") " pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.501593 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.501569    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed7719e2-3efb-4ab7-8912-965c94a54130-tls-cert\") pod \"authorino-6d4dfc649f-xfg6z\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") " pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.508603 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.508578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5nvb\" (UniqueName: \"kubernetes.io/projected/ed7719e2-3efb-4ab7-8912-965c94a54130-kube-api-access-n5nvb\") pod \"authorino-6d4dfc649f-xfg6z\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") " pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.600457 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.600409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bc2d132-7270-4660-bffc-ab06e9f3b0af-tls-cert\") pod \"authorino-69c4d5d9c9-lrzzp\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") " pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.600667 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.600500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7t8t\" (UniqueName: \"kubernetes.io/projected/9bc2d132-7270-4660-bffc-ab06e9f3b0af-kube-api-access-j7t8t\") pod \"authorino-69c4d5d9c9-lrzzp\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") " pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.602972 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.602945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bc2d132-7270-4660-bffc-ab06e9f3b0af-tls-cert\") pod \"authorino-69c4d5d9c9-lrzzp\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") " pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.610061 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.610035    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7t8t\" (UniqueName: \"kubernetes.io/projected/9bc2d132-7270-4660-bffc-ab06e9f3b0af-kube-api-access-j7t8t\") pod \"authorino-69c4d5d9c9-lrzzp\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") " pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.667592 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.667551    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:30:44.702258 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.702224    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.702466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.702224    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5m69n"
Sep 10 19:30:44.710271 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.710243    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:44.713578 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.713556    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5m69n"
Sep 10 19:30:44.802292 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.802196    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed7719e2-3efb-4ab7-8912-965c94a54130-tls-cert\") pod \"ed7719e2-3efb-4ab7-8912-965c94a54130\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") "
Sep 10 19:30:44.802446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.802296    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5nvb\" (UniqueName: \"kubernetes.io/projected/ed7719e2-3efb-4ab7-8912-965c94a54130-kube-api-access-n5nvb\") pod \"ed7719e2-3efb-4ab7-8912-965c94a54130\" (UID: \"ed7719e2-3efb-4ab7-8912-965c94a54130\") "
Sep 10 19:30:44.802446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.802403    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqbvf\" (UniqueName: \"kubernetes.io/projected/73756f49-468e-48d3-a1d8-3c3f87e39833-kube-api-access-zqbvf\") pod \"73756f49-468e-48d3-a1d8-3c3f87e39833\" (UID: \"73756f49-468e-48d3-a1d8-3c3f87e39833\") "
Sep 10 19:30:44.804567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.804530    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7719e2-3efb-4ab7-8912-965c94a54130-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ed7719e2-3efb-4ab7-8912-965c94a54130" (UID: "ed7719e2-3efb-4ab7-8912-965c94a54130"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:44.804567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.804558    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73756f49-468e-48d3-a1d8-3c3f87e39833-kube-api-access-zqbvf" (OuterVolumeSpecName: "kube-api-access-zqbvf") pod "73756f49-468e-48d3-a1d8-3c3f87e39833" (UID: "73756f49-468e-48d3-a1d8-3c3f87e39833"). InnerVolumeSpecName "kube-api-access-zqbvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:44.804744 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.804585    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed7719e2-3efb-4ab7-8912-965c94a54130-kube-api-access-n5nvb" (OuterVolumeSpecName: "kube-api-access-n5nvb") pod "ed7719e2-3efb-4ab7-8912-965c94a54130" (UID: "ed7719e2-3efb-4ab7-8912-965c94a54130"). InnerVolumeSpecName "kube-api-access-n5nvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:44.903485 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.903456    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqbvf\" (UniqueName: \"kubernetes.io/projected/73756f49-468e-48d3-a1d8-3c3f87e39833-kube-api-access-zqbvf\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:30:44.903485 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.903482    2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed7719e2-3efb-4ab7-8912-965c94a54130-tls-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:30:44.903485 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.903492    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5nvb\" (UniqueName: \"kubernetes.io/projected/ed7719e2-3efb-4ab7-8912-965c94a54130-kube-api-access-n5nvb\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:30:44.940252 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:44.940227    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69c4d5d9c9-lrzzp"]
Sep 10 19:30:44.942609 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:30:44.942581    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bc2d132_7270_4660_bffc_ab06e9f3b0af.slice/crio-105698d41ae246dfefc4a46055f335109d4fe1d2469e417be74eb118b962fb35 WatchSource:0}: Error finding container 105698d41ae246dfefc4a46055f335109d4fe1d2469e417be74eb118b962fb35: Status 404 returned error can't find the container with id 105698d41ae246dfefc4a46055f335109d4fe1d2469e417be74eb118b962fb35
Sep 10 19:30:45.713449 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:45.713420    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5m69n"
Sep 10 19:30:45.713592 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:45.713448    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp" event={"ID":"9bc2d132-7270-4660-bffc-ab06e9f3b0af","Type":"ContainerStarted","Data":"105698d41ae246dfefc4a46055f335109d4fe1d2469e417be74eb118b962fb35"}
Sep 10 19:30:45.713644 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:45.713631    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-xfg6z"
Sep 10 19:30:45.753448 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:45.753413    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-5m69n"]
Sep 10 19:30:45.755380 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:45.755346    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-5m69n"]
Sep 10 19:30:45.777281 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:45.777239    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-xfg6z"]
Sep 10 19:30:45.783226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:45.783187    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-xfg6z"]
Sep 10 19:30:46.212717 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:46.212655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:46.719350 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:46.719312    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp" event={"ID":"9bc2d132-7270-4660-bffc-ab06e9f3b0af","Type":"ContainerStarted","Data":"962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10"}
Sep 10 19:30:46.736715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:46.736649    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp" podStartSLOduration=2.014582622 podStartE2EDuration="2.73663161s" totalImagesPullingTime="722.048988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:46.736184609 +0000 UTC m=+639.676218442" watchObservedRunningTime="2026-09-10 19:30:46.73663161 +0000 UTC m=+639.676665443"
Sep 10 19:30:46.763452 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:46.763417    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5spv7"]
Sep 10 19:30:46.763643 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:46.763620    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-5spv7" podUID="1cf860dc-a732-49b9-a604-23f5c0dbb509" containerName="authorino" containerID="cri-o://8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d" gracePeriod=30
Sep 10 19:30:47.021748 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.021720    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5spv7"
Sep 10 19:30:47.122223 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.122184    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpdjg\" (UniqueName: \"kubernetes.io/projected/1cf860dc-a732-49b9-a604-23f5c0dbb509-kube-api-access-jpdjg\") pod \"1cf860dc-a732-49b9-a604-23f5c0dbb509\" (UID: \"1cf860dc-a732-49b9-a604-23f5c0dbb509\") "
Sep 10 19:30:47.124401 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.124365    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf860dc-a732-49b9-a604-23f5c0dbb509-kube-api-access-jpdjg" (OuterVolumeSpecName: "kube-api-access-jpdjg") pod "1cf860dc-a732-49b9-a604-23f5c0dbb509" (UID: "1cf860dc-a732-49b9-a604-23f5c0dbb509"). InnerVolumeSpecName "kube-api-access-jpdjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:47.223726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.223683    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpdjg\" (UniqueName: \"kubernetes.io/projected/1cf860dc-a732-49b9-a604-23f5c0dbb509-kube-api-access-jpdjg\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:30:47.694967 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.694933    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73756f49-468e-48d3-a1d8-3c3f87e39833" path="/var/lib/kubelet/pods/73756f49-468e-48d3-a1d8-3c3f87e39833/volumes"
Sep 10 19:30:47.695205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.695193    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed7719e2-3efb-4ab7-8912-965c94a54130" path="/var/lib/kubelet/pods/ed7719e2-3efb-4ab7-8912-965c94a54130/volumes"
Sep 10 19:30:47.725276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.725244    2562 generic.go:356] "Generic (PLEG): container finished" podID="1cf860dc-a732-49b9-a604-23f5c0dbb509" containerID="8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d" exitCode=0
Sep 10 19:30:47.725452 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.725284    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-5spv7"
Sep 10 19:30:47.725452 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.725331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5spv7" event={"ID":"1cf860dc-a732-49b9-a604-23f5c0dbb509","Type":"ContainerDied","Data":"8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d"}
Sep 10 19:30:47.725452 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.725384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-5spv7" event={"ID":"1cf860dc-a732-49b9-a604-23f5c0dbb509","Type":"ContainerDied","Data":"19712439c1e530588b78850e4ed7594c9bd12fa8e65363ae7ff543b85c8b46ae"}
Sep 10 19:30:47.725452 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.725416    2562 scope.go:117] "RemoveContainer" containerID="8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d"
Sep 10 19:30:47.733927 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.733904    2562 scope.go:117] "RemoveContainer" containerID="8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d"
Sep 10 19:30:47.734245 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:30:47.734217    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d\": container with ID starting with 8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d not found: ID does not exist" containerID="8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d"
Sep 10 19:30:47.734301 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.734257    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d"} err="failed to get container status \"8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d\": rpc error: code = NotFound desc = could not find container \"8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d\": container with ID starting with 8ce6e45bfaf3f56379bccee9e1ed271ef364539b4af95155b9831dbbd9c1010d not found: ID does not exist"
Sep 10 19:30:47.743683 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.743656    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5spv7"]
Sep 10 19:30:47.749633 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:47.749608    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-5spv7"]
Sep 10 19:30:49.695714 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:49.695674    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cf860dc-a732-49b9-a604-23f5c0dbb509" path="/var/lib/kubelet/pods/1cf860dc-a732-49b9-a604-23f5c0dbb509/volumes"
Sep 10 19:30:51.812348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:51.812314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:51.908882 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:51.908846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:52.410226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:52.410188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:55.258095 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:55.258053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:55.886138 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:55.886099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:30:55.981503 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:30:55.981466    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:09.640865 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:09.640821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:09.740254 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:09.740218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:27.465524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:27.465485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:27.554377 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:27.554346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:29.018398 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:29.018356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:29.116171 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:29.116130    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:29.243481 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:29.243428    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:29.711364 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:29.711331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:29.895154 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:29.895115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:30.004120 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:30.004031    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:31:32.663600 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:32.663567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:39.898664 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:39.898624    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:39.995469 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:39.995426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:45.365387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:45.365350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:51.015830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:51.015789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:51.540032 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:51.539990    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:31:56.318242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:56.318196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:31:57.094898 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:31:57.094844    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:32:02.287574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:02.287488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:03.194666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:03.194625    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:32:20.084743 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.084707    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"]
Sep 10 19:32:20.085227 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.085104    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cf860dc-a732-49b9-a604-23f5c0dbb509" containerName="authorino"
Sep 10 19:32:20.085227 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.085117    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf860dc-a732-49b9-a604-23f5c0dbb509" containerName="authorino"
Sep 10 19:32:20.085227 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.085189    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cf860dc-a732-49b9-a604-23f5c0dbb509" containerName="authorino"
Sep 10 19:32:20.088210 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.088188    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.092696 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.092660    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 19:32:20.094361 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.094335    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 19:32:20.094495 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.094378    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 19:32:20.094495 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.094396    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xklpr\""
Sep 10 19:32:20.100573 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.100543    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"]
Sep 10 19:32:20.124846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.124809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:20.261592 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.261542    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.261794 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.261654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.261794 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.261706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.261794 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.261779    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.261987 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.261971    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.262028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.262012    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fs42\" (UniqueName: \"kubernetes.io/projected/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-kube-api-access-7fs42\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.363497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.363401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.363497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.363452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.363497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.363476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs42\" (UniqueName: \"kubernetes.io/projected/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-kube-api-access-7fs42\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.363795 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.363548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.363795 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.363577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.363795 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.363602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.363989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.363823    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.364031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.364009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.364069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.364033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.365913 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.365862    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.366081 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.366062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.377382 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.377349    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fs42\" (UniqueName: \"kubernetes.io/projected/7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a-kube-api-access-7fs42\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kscj\" (UID: \"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.398393 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.398354    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:20.675360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:20.675332    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"]
Sep 10 19:32:20.677923 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:32:20.677879    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c06fd0f_d1d9_4a0c_bd2e_204cfcacfd0a.slice/crio-40765107f3f517c3ba4c8f8d8c9f7aa76e13bf31db3b0d3f86f9239baaea10e4 WatchSource:0}: Error finding container 40765107f3f517c3ba4c8f8d8c9f7aa76e13bf31db3b0d3f86f9239baaea10e4: Status 404 returned error can't find the container with id 40765107f3f517c3ba4c8f8d8c9f7aa76e13bf31db3b0d3f86f9239baaea10e4
Sep 10 19:32:21.164069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:21.164006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj" event={"ID":"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a","Type":"ContainerStarted","Data":"40765107f3f517c3ba4c8f8d8c9f7aa76e13bf31db3b0d3f86f9239baaea10e4"}
Sep 10 19:32:21.314612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:21.313822    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:32:22.170495 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:22.170455    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj" event={"ID":"7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a","Type":"ContainerStarted","Data":"4f1f58e67fc1fc5b4685a71f495bb952d59bb04220a5616b4897122a880c547a"}
Sep 10 19:32:22.194944 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:22.194864    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj" podStartSLOduration=1.083921004 podStartE2EDuration="2.194844272s" totalImagesPullingTime="1.110923268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:22.193701606 +0000 UTC m=+735.133735453" watchObservedRunningTime="2026-09-10 19:32:22.194844272 +0000 UTC m=+735.134878105"
Sep 10 19:32:29.910322 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:29.910280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:30.399576 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:30.399534    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:30.399752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:30.399606    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:30.412365 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:30.412334    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:30.917609 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:30.917571    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:32:31.234299 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:31.234205    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kscj"
Sep 10 19:32:32.666737 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.666692    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"]
Sep 10 19:32:32.670515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.670490    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.673824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.673796    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 19:32:32.676691 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.676657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.676824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.676719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.676824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.676783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.676981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.676847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.676981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.676902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66ck7\" (UniqueName: \"kubernetes.io/projected/c22612dd-8653-4332-b7ae-b595287342a0-kube-api-access-66ck7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.676981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.676939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22612dd-8653-4332-b7ae-b595287342a0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.682037 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.682010    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"]
Sep 10 19:32:32.778312 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.778508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.778508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.778508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66ck7\" (UniqueName: \"kubernetes.io/projected/c22612dd-8653-4332-b7ae-b595287342a0-kube-api-access-66ck7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.778508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22612dd-8653-4332-b7ae-b595287342a0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.778732 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.778732 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.778842 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.778825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.779099 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.779078    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.780727 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.780705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22612dd-8653-4332-b7ae-b595287342a0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.781036 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.781017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22612dd-8653-4332-b7ae-b595287342a0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.788107 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.788086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66ck7\" (UniqueName: \"kubernetes.io/projected/c22612dd-8653-4332-b7ae-b595287342a0-kube-api-access-66ck7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dvjcp\" (UID: \"c22612dd-8653-4332-b7ae-b595287342a0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:32.792632 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.792599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:32.980531 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:32.980429    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:33.253829 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:33.253733    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"]
Sep 10 19:32:33.257230 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:32:33.257199    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22612dd_8653_4332_b7ae_b595287342a0.slice/crio-f8693ae0351d16058d9208194bbe1d8def3233198cf9f328952d47a0f27f7a22 WatchSource:0}: Error finding container f8693ae0351d16058d9208194bbe1d8def3233198cf9f328952d47a0f27f7a22: Status 404 returned error can't find the container with id f8693ae0351d16058d9208194bbe1d8def3233198cf9f328952d47a0f27f7a22
Sep 10 19:32:33.258946 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:33.258927    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:32:34.232957 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:34.232916    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp" event={"ID":"c22612dd-8653-4332-b7ae-b595287342a0","Type":"ContainerStarted","Data":"af10417965e73ad439f0028b7dab9eb5bc0dfc618617057f2abf86fc6c3f4430"}
Sep 10 19:32:34.233441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:34.232976    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp" event={"ID":"c22612dd-8653-4332-b7ae-b595287342a0","Type":"ContainerStarted","Data":"f8693ae0351d16058d9208194bbe1d8def3233198cf9f328952d47a0f27f7a22"}
Sep 10 19:32:34.256345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:34.256284    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp" podStartSLOduration=2.061534535 podStartE2EDuration="2.256264993s" totalImagesPullingTime="194.730458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:34.253777913 +0000 UTC m=+747.193811746" watchObservedRunningTime="2026-09-10 19:32:34.256264993 +0000 UTC m=+747.196298826"
Sep 10 19:32:38.217433 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:38.217390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:32:39.510752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:39.510717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:32:42.980861 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:42.980823    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:42.981386 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:42.980928    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:42.993559 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:42.993522    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:43.286589 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:43.286509    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dvjcp"
Sep 10 19:32:46.109124 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:32:46.109082    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:33:56.380172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.380132    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c5578f46b-w9q8q"]
Sep 10 19:33:56.383829 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.383810    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.397247 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.397221    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5578f46b-w9q8q"]
Sep 10 19:33:56.554547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.554510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbp4n\" (UniqueName: \"kubernetes.io/projected/9d8adf50-e1d5-49de-8b32-ed5f44e5f382-kube-api-access-zbp4n\") pod \"authorino-7c5578f46b-w9q8q\" (UID: \"9d8adf50-e1d5-49de-8b32-ed5f44e5f382\") " pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.554734 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.554613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d8adf50-e1d5-49de-8b32-ed5f44e5f382-tls-cert\") pod \"authorino-7c5578f46b-w9q8q\" (UID: \"9d8adf50-e1d5-49de-8b32-ed5f44e5f382\") " pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.655921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.655870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbp4n\" (UniqueName: \"kubernetes.io/projected/9d8adf50-e1d5-49de-8b32-ed5f44e5f382-kube-api-access-zbp4n\") pod \"authorino-7c5578f46b-w9q8q\" (UID: \"9d8adf50-e1d5-49de-8b32-ed5f44e5f382\") " pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.656105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.655961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d8adf50-e1d5-49de-8b32-ed5f44e5f382-tls-cert\") pod \"authorino-7c5578f46b-w9q8q\" (UID: \"9d8adf50-e1d5-49de-8b32-ed5f44e5f382\") " pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.658409 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.658385    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9d8adf50-e1d5-49de-8b32-ed5f44e5f382-tls-cert\") pod \"authorino-7c5578f46b-w9q8q\" (UID: \"9d8adf50-e1d5-49de-8b32-ed5f44e5f382\") " pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.664964 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.664935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbp4n\" (UniqueName: \"kubernetes.io/projected/9d8adf50-e1d5-49de-8b32-ed5f44e5f382-kube-api-access-zbp4n\") pod \"authorino-7c5578f46b-w9q8q\" (UID: \"9d8adf50-e1d5-49de-8b32-ed5f44e5f382\") " pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.693343 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.693307    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c5578f46b-w9q8q"
Sep 10 19:33:56.966708 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:56.966682    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c5578f46b-w9q8q"]
Sep 10 19:33:56.968800 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:33:56.968765    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d8adf50_e1d5_49de_8b32_ed5f44e5f382.slice/crio-4f8f54d640d665b44e525501a0c54d3be086381dc0b8e081e9c55715fb7846f4 WatchSource:0}: Error finding container 4f8f54d640d665b44e525501a0c54d3be086381dc0b8e081e9c55715fb7846f4: Status 404 returned error can't find the container with id 4f8f54d640d665b44e525501a0c54d3be086381dc0b8e081e9c55715fb7846f4
Sep 10 19:33:57.624692 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:57.624645    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5578f46b-w9q8q" event={"ID":"9d8adf50-e1d5-49de-8b32-ed5f44e5f382","Type":"ContainerStarted","Data":"4f8f54d640d665b44e525501a0c54d3be086381dc0b8e081e9c55715fb7846f4"}
Sep 10 19:33:58.630765 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:58.630724    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c5578f46b-w9q8q" event={"ID":"9d8adf50-e1d5-49de-8b32-ed5f44e5f382","Type":"ContainerStarted","Data":"b92d188b56c4a9783b6d409a6d0c202c88897592616f1a1577965178255cbbc2"}
Sep 10 19:33:58.664196 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:58.664140    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c5578f46b-w9q8q" podStartSLOduration=1.7508107499999999 podStartE2EDuration="2.664123694s" totalImagesPullingTime="913.312944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:58.661878451 +0000 UTC m=+831.601912280" watchObservedRunningTime="2026-09-10 19:33:58.664123694 +0000 UTC m=+831.604157526"
Sep 10 19:33:58.714337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:58.714301    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69c4d5d9c9-lrzzp"]
Sep 10 19:33:58.714555 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:58.714526    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp" podUID="9bc2d132-7270-4660-bffc-ab06e9f3b0af" containerName="authorino" containerID="cri-o://962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10" gracePeriod=30
Sep 10 19:33:58.968463 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:58.968434    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:33:59.080822 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.080776    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bc2d132-7270-4660-bffc-ab06e9f3b0af-tls-cert\") pod \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") "
Sep 10 19:33:59.081040 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.080843    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7t8t\" (UniqueName: \"kubernetes.io/projected/9bc2d132-7270-4660-bffc-ab06e9f3b0af-kube-api-access-j7t8t\") pod \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\" (UID: \"9bc2d132-7270-4660-bffc-ab06e9f3b0af\") "
Sep 10 19:33:59.084907 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.083671    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc2d132-7270-4660-bffc-ab06e9f3b0af-kube-api-access-j7t8t" (OuterVolumeSpecName: "kube-api-access-j7t8t") pod "9bc2d132-7270-4660-bffc-ab06e9f3b0af" (UID: "9bc2d132-7270-4660-bffc-ab06e9f3b0af"). InnerVolumeSpecName "kube-api-access-j7t8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:59.094613 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.094570    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bc2d132-7270-4660-bffc-ab06e9f3b0af-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9bc2d132-7270-4660-bffc-ab06e9f3b0af" (UID: "9bc2d132-7270-4660-bffc-ab06e9f3b0af"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:33:59.182831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.182788    2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bc2d132-7270-4660-bffc-ab06e9f3b0af-tls-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:33:59.183017 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.182838    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7t8t\" (UniqueName: \"kubernetes.io/projected/9bc2d132-7270-4660-bffc-ab06e9f3b0af-kube-api-access-j7t8t\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:33:59.431102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.431059    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:59.623320 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.623215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:33:59.638096 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.638051    2562 generic.go:356] "Generic (PLEG): container finished" podID="9bc2d132-7270-4660-bffc-ab06e9f3b0af" containerID="962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10" exitCode=0
Sep 10 19:33:59.638714 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.638689    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp"
Sep 10 19:33:59.641969 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.641932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp" event={"ID":"9bc2d132-7270-4660-bffc-ab06e9f3b0af","Type":"ContainerDied","Data":"962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10"}
Sep 10 19:33:59.642125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.641993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c4d5d9c9-lrzzp" event={"ID":"9bc2d132-7270-4660-bffc-ab06e9f3b0af","Type":"ContainerDied","Data":"105698d41ae246dfefc4a46055f335109d4fe1d2469e417be74eb118b962fb35"}
Sep 10 19:33:59.642125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.642014    2562 scope.go:117] "RemoveContainer" containerID="962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10"
Sep 10 19:33:59.662512 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.662487    2562 scope.go:117] "RemoveContainer" containerID="962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10"
Sep 10 19:33:59.662955 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:33:59.662926    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10\": container with ID starting with 962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10 not found: ID does not exist" containerID="962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10"
Sep 10 19:33:59.663065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.662967    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10"} err="failed to get container status \"962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10\": rpc error: code = NotFound desc = could not find container \"962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10\": container with ID starting with 962067da84aaac9d0415c47ae5b612cfd0fec94fc313ba6d8a6af061df8f8e10 not found: ID does not exist"
Sep 10 19:33:59.687835 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.687794    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69c4d5d9c9-lrzzp"]
Sep 10 19:33:59.695738 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:33:59.695711    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69c4d5d9c9-lrzzp"]
Sep 10 19:34:00.787346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:00.787309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:00.936703 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:00.936655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:01.695612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:01.695580    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bc2d132-7270-4660-bffc-ab06e9f3b0af" path="/var/lib/kubelet/pods/9bc2d132-7270-4660-bffc-ab06e9f3b0af/volumes"
Sep 10 19:34:18.748958 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:18.748915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:18.907508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:18.907465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:19.175269 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:19.175227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:19.306385 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:19.306349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:20.279283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.279249    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:20.473055 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.473014    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:20.794186 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.794147    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:20.963326 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.963290    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7745-de38fe"]
Sep 10 19:34:20.963666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.963652    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc2d132-7270-4660-bffc-ab06e9f3b0af" containerName="authorino"
Sep 10 19:34:20.963666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.963666    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc2d132-7270-4660-bffc-ab06e9f3b0af" containerName="authorino"
Sep 10 19:34:20.963754 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.963739    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bc2d132-7270-4660-bffc-ab06e9f3b0af" containerName="authorino"
Sep 10 19:34:20.968261 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.968242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-de38fe"
Sep 10 19:34:20.971210 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.971186    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zp7h9\""
Sep 10 19:34:20.976546 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:20.976522    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7745-de38fe"]
Sep 10 19:34:21.081456 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:21.081367    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d957\" (UniqueName: \"kubernetes.io/projected/acc244cd-8c61-4cbf-8d6f-9698f578d0d7-kube-api-access-4d957\") pod \"test-curl-7745-de38fe\" (UID: \"acc244cd-8c61-4cbf-8d6f-9698f578d0d7\") " pod="opendatahub/test-curl-7745-de38fe"
Sep 10 19:34:21.182727 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:21.182683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d957\" (UniqueName: \"kubernetes.io/projected/acc244cd-8c61-4cbf-8d6f-9698f578d0d7-kube-api-access-4d957\") pod \"test-curl-7745-de38fe\" (UID: \"acc244cd-8c61-4cbf-8d6f-9698f578d0d7\") " pod="opendatahub/test-curl-7745-de38fe"
Sep 10 19:34:21.193385 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:21.193361    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d957\" (UniqueName: \"kubernetes.io/projected/acc244cd-8c61-4cbf-8d6f-9698f578d0d7-kube-api-access-4d957\") pod \"test-curl-7745-de38fe\" (UID: \"acc244cd-8c61-4cbf-8d6f-9698f578d0d7\") " pod="opendatahub/test-curl-7745-de38fe"
Sep 10 19:34:21.278576 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:21.278539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-de38fe"
Sep 10 19:34:21.553414 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:21.553379    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7745-de38fe"]
Sep 10 19:34:21.557218 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:34:21.557174    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacc244cd_8c61_4cbf_8d6f_9698f578d0d7.slice/crio-e86dfebe4dc2b486f2a45458e75aacaa295566cd2e80fa9a8d1b3bb81fd98b08 WatchSource:0}: Error finding container e86dfebe4dc2b486f2a45458e75aacaa295566cd2e80fa9a8d1b3bb81fd98b08: Status 404 returned error can't find the container with id e86dfebe4dc2b486f2a45458e75aacaa295566cd2e80fa9a8d1b3bb81fd98b08
Sep 10 19:34:21.741115 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:21.741079    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7745-de38fe" event={"ID":"acc244cd-8c61-4cbf-8d6f-9698f578d0d7","Type":"ContainerStarted","Data":"e86dfebe4dc2b486f2a45458e75aacaa295566cd2e80fa9a8d1b3bb81fd98b08"}
Sep 10 19:34:22.366160 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:22.366119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:22.467291 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:22.467252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:22.518798 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:22.518760    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:34:23.569422 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:23.569298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:23.640917 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:23.640839    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:23.748370 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:23.748316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:24.050031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:24.050000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:24.118244 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:24.118205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:24.758074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:24.758036    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7745-de38fe" event={"ID":"acc244cd-8c61-4cbf-8d6f-9698f578d0d7","Type":"ContainerStarted","Data":"0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f"}
Sep 10 19:34:24.775181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:24.775119    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7745-de38fe" podStartSLOduration=2.381309296 podStartE2EDuration="4.775101182s" totalImagesPullingTime="2.393791886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:24.772963209 +0000 UTC m=+857.712997042" watchObservedRunningTime="2026-09-10 19:34:24.775101182 +0000 UTC m=+857.715135015"
Sep 10 19:34:25.397286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.397247    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7745-de38fe"]
Sep 10 19:34:25.399078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.399046    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7745-de38fe"]
Sep 10 19:34:25.514609 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.514518    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7745-882816"]
Sep 10 19:34:25.515034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.515014    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acc244cd-8c61-4cbf-8d6f-9698f578d0d7" containerName="test-curl-7745-de38fe"
Sep 10 19:34:25.515148 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.515036    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="acc244cd-8c61-4cbf-8d6f-9698f578d0d7" containerName="test-curl-7745-de38fe"
Sep 10 19:34:25.515148 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.515111    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="acc244cd-8c61-4cbf-8d6f-9698f578d0d7" containerName="test-curl-7745-de38fe"
Sep 10 19:34:25.518296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.518274    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-882816"
Sep 10 19:34:25.529332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.529302    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7745-882816"]
Sep 10 19:34:25.623239 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.623195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvtxw\" (UniqueName: \"kubernetes.io/projected/df5388a8-5a23-4ec4-98e4-7a6526ad764c-kube-api-access-jvtxw\") pod \"test-curl-7745-882816\" (UID: \"df5388a8-5a23-4ec4-98e4-7a6526ad764c\") " pod="opendatahub/test-curl-7745-882816"
Sep 10 19:34:25.724348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.724304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvtxw\" (UniqueName: \"kubernetes.io/projected/df5388a8-5a23-4ec4-98e4-7a6526ad764c-kube-api-access-jvtxw\") pod \"test-curl-7745-882816\" (UID: \"df5388a8-5a23-4ec4-98e4-7a6526ad764c\") " pod="opendatahub/test-curl-7745-882816"
Sep 10 19:34:25.732360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.732331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvtxw\" (UniqueName: \"kubernetes.io/projected/df5388a8-5a23-4ec4-98e4-7a6526ad764c-kube-api-access-jvtxw\") pod \"test-curl-7745-882816\" (UID: \"df5388a8-5a23-4ec4-98e4-7a6526ad764c\") " pod="opendatahub/test-curl-7745-882816"
Sep 10 19:34:25.762425 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:34:25.762393    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7745-de38fe" container="test-curl-7745-de38fe"
Sep 10 19:34:25.827577 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:25.827492    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-882816"
Sep 10 19:34:26.097205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:26.097179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7745-882816"]
Sep 10 19:34:26.099301 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:34:26.099269    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf5388a8_5a23_4ec4_98e4_7a6526ad764c.slice/crio-d845ea2dd500ac1c0c9f9d1ea38b1f45dba25b8b2b2ddf84163f63a6d9d9fe83 WatchSource:0}: Error finding container d845ea2dd500ac1c0c9f9d1ea38b1f45dba25b8b2b2ddf84163f63a6d9d9fe83: Status 404 returned error can't find the container with id d845ea2dd500ac1c0c9f9d1ea38b1f45dba25b8b2b2ddf84163f63a6d9d9fe83
Sep 10 19:34:26.769185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:26.769147    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7745-882816" event={"ID":"df5388a8-5a23-4ec4-98e4-7a6526ad764c","Type":"ContainerStarted","Data":"0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799"}
Sep 10 19:34:26.769185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:26.769192    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7745-882816" event={"ID":"df5388a8-5a23-4ec4-98e4-7a6526ad764c","Type":"ContainerStarted","Data":"d845ea2dd500ac1c0c9f9d1ea38b1f45dba25b8b2b2ddf84163f63a6d9d9fe83"}
Sep 10 19:34:26.769656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:26.769292    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7745-de38fe" podUID="acc244cd-8c61-4cbf-8d6f-9698f578d0d7" containerName="test-curl-7745-de38fe" containerID="cri-o://0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f" gracePeriod=2
Sep 10 19:34:26.784869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:26.784814    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7745-882816" podStartSLOduration=1.784797932 podStartE2EDuration="1.784797932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:26.783019635 +0000 UTC m=+859.723053468" watchObservedRunningTime="2026-09-10 19:34:26.784797932 +0000 UTC m=+859.724831766"
Sep 10 19:34:27.275966 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:27.275926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:27.357662 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:27.357622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:27.400543 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:27.400506    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7745-882816"]
Sep 10 19:34:27.405017 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:27.404983    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7745-882816"]
Sep 10 19:34:27.523483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:27.523385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:27.774016 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:34:27.773937    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7745-882816" container="test-curl-7745-882816"
Sep 10 19:34:27.972913 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:27.972861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:28.073124 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.073018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:28.172744 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.172708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:28.338697 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.338615    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"]
Sep 10 19:34:28.339173 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.339154    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5388a8-5a23-4ec4-98e4-7a6526ad764c" containerName="test-curl-7745-882816"
Sep 10 19:34:28.339224 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.339174    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5388a8-5a23-4ec4-98e4-7a6526ad764c" containerName="test-curl-7745-882816"
Sep 10 19:34:28.339261 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.339253    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5388a8-5a23-4ec4-98e4-7a6526ad764c" containerName="test-curl-7745-882816"
Sep 10 19:34:28.342430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.342412    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.345100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.345075    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 19:34:28.345213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.345075    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-18c3c3-openshift-default-dockercfg-s9dhk\""
Sep 10 19:34:28.372875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.372842    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"]
Sep 10 19:34:28.451707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.451673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxhp2\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-kube-api-access-hxhp2\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.451707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.451715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-podinfo\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.452008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.451762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-certs\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.452008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.451776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-token\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.452008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.451868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istiod-ca-cert\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.452008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.451957    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-data\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.452008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.451984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-envoy\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.452008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.452009    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-socket\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.452214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.452030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-credential-socket\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.455260 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.455227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:28.553155 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-certs\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553155 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-token\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istiod-ca-cert\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-data\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553372    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-envoy\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-socket\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-credential-socket\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhp2\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-kube-api-access-hxhp2\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-podinfo\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-certs\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553927 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-credential-socket\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.553986 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.553966    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-data\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.554029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.554010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-socket\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.554300 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.554273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istiod-ca-cert\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.555752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.555719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-envoy\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.556171 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.556150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-podinfo\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.576337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.576306    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxhp2\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-kube-api-access-hxhp2\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.576479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.576305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-token\") pod \"e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") " pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.585226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.585183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:28.652789 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.652752    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:28.779383 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.779330    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7745-882816" podUID="df5388a8-5a23-4ec4-98e4-7a6526ad764c" containerName="test-curl-7745-882816" containerID="cri-o://0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799" gracePeriod=2
Sep 10 19:34:28.951070 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.951041    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"]
Sep 10 19:34:28.953102 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:34:28.953071    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a01fed9_04cd_47a1_837d_bf4d5c820bc8.slice/crio-f7c4e98c67208d4d07bc919deb431639d004504ed0f71999f3a1b81c06c3dec4 WatchSource:0}: Error finding container f7c4e98c67208d4d07bc919deb431639d004504ed0f71999f3a1b81c06c3dec4: Status 404 returned error can't find the container with id f7c4e98c67208d4d07bc919deb431639d004504ed0f71999f3a1b81c06c3dec4
Sep 10 19:34:28.968952 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:28.968919    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-de38fe"
Sep 10 19:34:29.058056 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.058018    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d957\" (UniqueName: \"kubernetes.io/projected/acc244cd-8c61-4cbf-8d6f-9698f578d0d7-kube-api-access-4d957\") pod \"acc244cd-8c61-4cbf-8d6f-9698f578d0d7\" (UID: \"acc244cd-8c61-4cbf-8d6f-9698f578d0d7\") "
Sep 10 19:34:29.060310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.060277    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acc244cd-8c61-4cbf-8d6f-9698f578d0d7-kube-api-access-4d957" (OuterVolumeSpecName: "kube-api-access-4d957") pod "acc244cd-8c61-4cbf-8d6f-9698f578d0d7" (UID: "acc244cd-8c61-4cbf-8d6f-9698f578d0d7"). InnerVolumeSpecName "kube-api-access-4d957". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:29.158966 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.158863    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d957\" (UniqueName: \"kubernetes.io/projected/acc244cd-8c61-4cbf-8d6f-9698f578d0d7-kube-api-access-4d957\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:34:29.697865 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.697823    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acc244cd-8c61-4cbf-8d6f-9698f578d0d7" path="/var/lib/kubelet/pods/acc244cd-8c61-4cbf-8d6f-9698f578d0d7/volumes"
Sep 10 19:34:29.786315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.786280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" event={"ID":"1a01fed9-04cd-47a1-837d-bf4d5c820bc8","Type":"ContainerStarted","Data":"f7c4e98c67208d4d07bc919deb431639d004504ed0f71999f3a1b81c06c3dec4"}
Sep 10 19:34:29.787819 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.787787    2562 generic.go:356] "Generic (PLEG): container finished" podID="acc244cd-8c61-4cbf-8d6f-9698f578d0d7" containerID="0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f" exitCode=137
Sep 10 19:34:29.787994 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.787869    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-de38fe"
Sep 10 19:34:29.787994 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.787879    2562 scope.go:117] "RemoveContainer" containerID="0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f"
Sep 10 19:34:29.797664 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.797639    2562 scope.go:117] "RemoveContainer" containerID="0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f"
Sep 10 19:34:29.798100 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:34:29.798073    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f\": container with ID starting with 0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f not found: ID does not exist" containerID="0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f"
Sep 10 19:34:29.798186 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:29.798113    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f"} err="failed to get container status \"0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f\": rpc error: code = NotFound desc = could not find container \"0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f\": container with ID starting with 0d823908c9636172610289df1232c2abe5807bbfbc4c711c41581d52799fa18f not found: ID does not exist"
Sep 10 19:34:31.593311 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.593257    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:31.593618 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.593368    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:31.593618 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.593416    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:31.690387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.690360    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-882816"
Sep 10 19:34:31.786841 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.786740    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvtxw\" (UniqueName: \"kubernetes.io/projected/df5388a8-5a23-4ec4-98e4-7a6526ad764c-kube-api-access-jvtxw\") pod \"df5388a8-5a23-4ec4-98e4-7a6526ad764c\" (UID: \"df5388a8-5a23-4ec4-98e4-7a6526ad764c\") "
Sep 10 19:34:31.789066 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.789038    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df5388a8-5a23-4ec4-98e4-7a6526ad764c-kube-api-access-jvtxw" (OuterVolumeSpecName: "kube-api-access-jvtxw") pod "df5388a8-5a23-4ec4-98e4-7a6526ad764c" (UID: "df5388a8-5a23-4ec4-98e4-7a6526ad764c"). InnerVolumeSpecName "kube-api-access-jvtxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:31.802729 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.802694    2562 generic.go:356] "Generic (PLEG): container finished" podID="df5388a8-5a23-4ec4-98e4-7a6526ad764c" containerID="0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799" exitCode=137
Sep 10 19:34:31.802923 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.802735    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7745-882816"
Sep 10 19:34:31.802923 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.802785    2562 scope.go:117] "RemoveContainer" containerID="0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799"
Sep 10 19:34:31.804628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.804599    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" event={"ID":"1a01fed9-04cd-47a1-837d-bf4d5c820bc8","Type":"ContainerStarted","Data":"fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786"}
Sep 10 19:34:31.813538 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.813518    2562 scope.go:117] "RemoveContainer" containerID="0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799"
Sep 10 19:34:31.813901 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:34:31.813861    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799\": container with ID starting with 0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799 not found: ID does not exist" containerID="0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799"
Sep 10 19:34:31.813977 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.813918    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799"} err="failed to get container status \"0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799\": rpc error: code = NotFound desc = could not find container \"0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799\": container with ID starting with 0ae1ea645439dd8b4a4a4ac4df0fb029ee7d3904b7ffc055b1b760ef95dcd799 not found: ID does not exist"
Sep 10 19:34:31.839654 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.839576    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" podStartSLOduration=1.202321314 podStartE2EDuration="3.839558173s" totalImagesPullingTime="2.637236859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:31.838164832 +0000 UTC m=+864.778198663" watchObservedRunningTime="2026-09-10 19:34:31.839558173 +0000 UTC m=+864.779592002"
Sep 10 19:34:31.888323 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:31.888094    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvtxw\" (UniqueName: \"kubernetes.io/projected/df5388a8-5a23-4ec4-98e4-7a6526ad764c-kube-api-access-jvtxw\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:34:32.653659 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:32.653616    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:32.658803 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:32.658774    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:32.810360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:32.810323    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:32.811524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:32.811506    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:33.429649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:33.429609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:33.614615 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:33.614580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:33.695483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:33.695383    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df5388a8-5a23-4ec4-98e4-7a6526ad764c" path="/var/lib/kubelet/pods/df5388a8-5a23-4ec4-98e4-7a6526ad764c/volumes"
Sep 10 19:34:33.952708 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:33.952610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:34.136752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:34.136710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:34.600829 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:34.600788    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:34.708845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:34.708810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:34.809258 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:34.809215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:35.190279 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:35.190248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:35.296287 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:35.296245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:35.648698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:35.648660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:35.725471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:35.725433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:35.812471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:35.812437    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:34:36.248880 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.248832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:36.348261 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.348223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:36.383211 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.383164    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"]
Sep 10 19:34:36.386996 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.386968    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.389549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.389522    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-18c3c3\""
Sep 10 19:34:36.389672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.389528    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-18c3c3-dockercfg-4gm85\""
Sep 10 19:34:36.397381 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.397356    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"]
Sep 10 19:34:36.536170 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.536068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqxw9\" (UniqueName: \"kubernetes.io/projected/2380d8f1-7459-4ebf-882b-3669411e7ac1-kube-api-access-cqxw9\") pod \"payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") " pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.536332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.536199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2380d8f1-7459-4ebf-882b-3669411e7ac1-plugins-config-volume\") pod \"payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") " pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.552997 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.552959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:36.637056 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.637020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2380d8f1-7459-4ebf-882b-3669411e7ac1-plugins-config-volume\") pod \"payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") " pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.637301 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.637088    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqxw9\" (UniqueName: \"kubernetes.io/projected/2380d8f1-7459-4ebf-882b-3669411e7ac1-kube-api-access-cqxw9\") pod \"payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") " pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.637993 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.637968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2380d8f1-7459-4ebf-882b-3669411e7ac1-plugins-config-volume\") pod \"payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") " pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.647875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.647835    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqxw9\" (UniqueName: \"kubernetes.io/projected/2380d8f1-7459-4ebf-882b-3669411e7ac1-kube-api-access-cqxw9\") pod \"payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") " pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.698298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.698256    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:36.864556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.864464    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:36.972572 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:36.972545    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"]
Sep 10 19:34:36.974363 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:34:36.974334    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2380d8f1_7459_4ebf_882b_3669411e7ac1.slice/crio-b0b979361b96cd2e085189c9934b0b734148ec7882fe92773b6c1b5d87f79bea WatchSource:0}: Error finding container b0b979361b96cd2e085189c9934b0b734148ec7882fe92773b6c1b5d87f79bea: Status 404 returned error can't find the container with id b0b979361b96cd2e085189c9934b0b734148ec7882fe92773b6c1b5d87f79bea
Sep 10 19:34:37.696664 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:37.696516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.818404 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:37.818349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:37.837525 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:37.837445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt" event={"ID":"2380d8f1-7459-4ebf-882b-3669411e7ac1","Type":"ContainerStarted","Data":"b0b979361b96cd2e085189c9934b0b734148ec7882fe92773b6c1b5d87f79bea"}
Sep 10 19:34:38.049698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:38.049574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:38.203310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:38.203272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:38.364387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:38.363931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:39.850207 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:39.850166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt" event={"ID":"2380d8f1-7459-4ebf-882b-3669411e7ac1","Type":"ContainerStarted","Data":"950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce"}
Sep 10 19:34:39.850620 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:39.850302    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:39.874222 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:39.874168    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt" podStartSLOduration=1.576006034 podStartE2EDuration="3.874143144s" totalImagesPullingTime="2.29813711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:39.872711896 +0000 UTC m=+872.812745712" watchObservedRunningTime="2026-09-10 19:34:39.874143144 +0000 UTC m=+872.814176971"
Sep 10 19:34:40.307333 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:40.307290    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:40.986770 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:40.986730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:42.371534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:42.371493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:42.481032 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:42.480995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:42.612583 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:42.612544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:42.719952 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:42.719915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:45.634899 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:45.634863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:47.267501 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:47.267460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:47.689699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:47.689662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:47.929635 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:47.929597    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"]
Sep 10 19:34:47.934037 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:47.934010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:47.936672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:47.936645    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a037c8-openshift-default-dockercfg-qd7sl\""
Sep 10 19:34:47.956789 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:47.956754    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"]
Sep 10 19:34:47.967402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:47.967368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:48.075586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.075538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-podinfo\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.075928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.075878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-socket\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.076122 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.076105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istiod-ca-cert\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.076283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.076268    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tv6b\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-kube-api-access-6tv6b\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.076421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.076405    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-envoy\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.076578 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.076564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-data\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.076719 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.076696    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-token\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.076866 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.076852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-credential-socket\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.077018 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.077003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-certs\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.177831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.177785    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-socket\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.177854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istiod-ca-cert\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.177914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tv6b\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-kube-api-access-6tv6b\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.177945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-envoy\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.177998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-data\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-token\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178061    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-credential-socket\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-certs\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-podinfo\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178209    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-socket\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178530 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178397    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-data\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178530 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178503    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istiod-ca-cert\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178673    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-credential-socket\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.178989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.178842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-certs\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.180682 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.180658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-podinfo\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.180909 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.180877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-envoy\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.191636 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.191602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-token\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.191787 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.191665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tv6b\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-kube-api-access-6tv6b\") pod \"e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") " pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.263810 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.263452    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:48.362917 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.361218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:48.557464 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:34:48.557414    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b863ade_c45e_45c0_8d00_d4cb4fff9546.slice/crio-cbde327869959adc69c0e1f01957d37c5608ce9a259c4888a0c036272747e7e8 WatchSource:0}: Error finding container cbde327869959adc69c0e1f01957d37c5608ce9a259c4888a0c036272747e7e8: Status 404 returned error can't find the container with id cbde327869959adc69c0e1f01957d37c5608ce9a259c4888a0c036272747e7e8
Sep 10 19:34:48.560400 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.560351    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:48.560507 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.560455    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:48.560570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.560502    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:48.565533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.565499    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"]
Sep 10 19:34:48.895812 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.895775    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" event={"ID":"0b863ade-c45e-45c0-8d00-d4cb4fff9546","Type":"ContainerStarted","Data":"5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c"}
Sep 10 19:34:48.896028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:48.895837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" event={"ID":"0b863ade-c45e-45c0-8d00-d4cb4fff9546","Type":"ContainerStarted","Data":"cbde327869959adc69c0e1f01957d37c5608ce9a259c4888a0c036272747e7e8"}
Sep 10 19:34:49.265220 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:49.265107    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:49.270388 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:49.270352    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:49.301724 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:49.301665    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podStartSLOduration=2.301647831 podStartE2EDuration="2.301647831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:48.929326344 +0000 UTC m=+881.869360178" watchObservedRunningTime="2026-09-10 19:34:49.301647831 +0000 UTC m=+882.241681664"
Sep 10 19:34:49.806588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:49.806548    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:34:49.900595 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:49.900561    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:49.901659 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:49.901634    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:34:50.118222 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:50.118190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:50.232152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:50.232118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:50.330259 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:50.330217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:50.488745 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:50.488711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:50.857427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:50.857345    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:34:50.918146 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:50.918108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:51.006391 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:51.006354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:52.249343 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:52.249272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:52.313918 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:52.313871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:52.744066 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:52.744030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:52.918948 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:52.918900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:53.155286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.123871    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"]
Sep 10 19:34:53.155286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.124101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:53.155286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.128269    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.155286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.131276    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a037c8-dockercfg-46gc6\""
Sep 10 19:34:53.155286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.131999    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a037c8\""
Sep 10 19:34:53.155286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.152350    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"]
Sep 10 19:34:53.234619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.234575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjdcq\" (UniqueName: \"kubernetes.io/projected/f058e745-d4b9-4044-bec6-168545fdd5a3-kube-api-access-bjdcq\") pod \"payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.234808 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.234783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f058e745-d4b9-4044-bec6-168545fdd5a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.263715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.263680    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"]
Sep 10 19:34:53.267238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.267216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.269977 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.269952    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-f4a65e71-kserve-self-signed-certs\""
Sep 10 19:34:53.277242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.277215    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"]
Sep 10 19:34:53.335929 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.335870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-home\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.335929 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.335930    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-model-cache\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.336205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.335953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmmbc\" (UniqueName: \"kubernetes.io/projected/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-kube-api-access-vmmbc\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.336205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.336016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tls-certs\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.336205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.336037    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-dshm\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.336205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.336098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tmp-dir\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.336205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.336190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f058e745-d4b9-4044-bec6-168545fdd5a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.336415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.336258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjdcq\" (UniqueName: \"kubernetes.io/projected/f058e745-d4b9-4044-bec6-168545fdd5a3-kube-api-access-bjdcq\") pod \"payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.336948 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.336929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f058e745-d4b9-4044-bec6-168545fdd5a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.345535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.345508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjdcq\" (UniqueName: \"kubernetes.io/projected/f058e745-d4b9-4044-bec6-168545fdd5a3-kube-api-access-bjdcq\") pod \"payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.437429 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tmp-dir\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-home\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-model-cache\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmmbc\" (UniqueName: \"kubernetes.io/projected/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-kube-api-access-vmmbc\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tls-certs\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-dshm\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tmp-dir\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437833    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-home\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.437970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.437928    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-model-cache\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.439874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.439853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-dshm\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.440144 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.440124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tls-certs\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.441989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.441969    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.446326 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.446266    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmmbc\" (UniqueName: \"kubernetes.io/projected/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-kube-api-access-vmmbc\") pod \"e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") " pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.573443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.573405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:53.602883 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.602842    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:34:53.727400 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.727353    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"]
Sep 10 19:34:53.878441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.878413    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"]
Sep 10 19:34:53.880413 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:34:53.880376    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a17b45b_bf92_4c3e_ad30_e0d0f169a5ef.slice/crio-9b38d0255e09f0070a17f64437ec093d3324c2943661a90b2d13baa68e6e5163 WatchSource:0}: Error finding container 9b38d0255e09f0070a17f64437ec093d3324c2943661a90b2d13baa68e6e5163: Status 404 returned error can't find the container with id 9b38d0255e09f0070a17f64437ec093d3324c2943661a90b2d13baa68e6e5163
Sep 10 19:34:53.922276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.922247    2562 generic.go:356] "Generic (PLEG): container finished" podID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerID="fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786" exitCode=0
Sep 10 19:34:53.922415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.922321    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" event={"ID":"1a01fed9-04cd-47a1-837d-bf4d5c820bc8","Type":"ContainerDied","Data":"fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786"}
Sep 10 19:34:53.922806 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.922784    2562 scope.go:117] "RemoveContainer" containerID="fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786"
Sep 10 19:34:53.923788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.923761    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl" event={"ID":"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef","Type":"ContainerStarted","Data":"9b38d0255e09f0070a17f64437ec093d3324c2943661a90b2d13baa68e6e5163"}
Sep 10 19:34:53.924087 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.924023    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:53.924176 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.924130    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:53.924242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.924179    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:34:53.925390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.925369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z" event={"ID":"f058e745-d4b9-4044-bec6-168545fdd5a3","Type":"ContainerStarted","Data":"28bd2bcc29484acdee5e0c7bde9c94e9751ad2e27918ab2e8b60b4fddbeb36fc"}
Sep 10 19:34:53.925466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.925408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z" event={"ID":"f058e745-d4b9-4044-bec6-168545fdd5a3","Type":"ContainerStarted","Data":"f3955a7fc5de37845a798e2b80bd1915316710eed9548c6d26dff6e276ce0fe1"}
Sep 10 19:34:53.925851 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.925837    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:34:53.997150 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:53.996983    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z" podStartSLOduration=0.996959736 podStartE2EDuration="996.959736ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:53.994313486 +0000 UTC m=+886.934347310" watchObservedRunningTime="2026-09-10 19:34:53.996959736 +0000 UTC m=+886.936993570"
Sep 10 19:34:54.932474 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:54.932437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" event={"ID":"1a01fed9-04cd-47a1-837d-bf4d5c820bc8","Type":"ContainerStarted","Data":"6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f"}
Sep 10 19:34:54.934213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:54.934180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl" event={"ID":"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef","Type":"ContainerStarted","Data":"7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af"}
Sep 10 19:34:54.998670 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:54.998584    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl" podStartSLOduration=1.998566114 podStartE2EDuration="1.998566114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:54.997639906 +0000 UTC m=+887.937673738" watchObservedRunningTime="2026-09-10 19:34:54.998566114 +0000 UTC m=+887.938599948"
Sep 10 19:34:55.653126 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:55.653087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:34:55.653350 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:55.653167    2562 patch_prober.go:28] interesting pod/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 19:34:55.653350 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:55.653231    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 19:34:55.783019 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:55.782985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:56.653788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:56.653741    2562 patch_prober.go:28] interesting pod/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 19:34:56.654860 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:56.654417    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 19:34:57.181576 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:57.181526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:57.284189 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:57.284155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:57.378224 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:57.378189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:57.653815 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:57.653774    2562 patch_prober.go:28] interesting pod/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 19:34:57.654427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:57.654386    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 19:34:58.148251 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:58.148204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:58.653719 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:58.653668    2562 patch_prober.go:28] interesting pod/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 19:34:58.653980 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:58.653752    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 19:34:58.995566 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:58.995445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:59.006637 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.006601    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"]
Sep 10 19:34:59.017404 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.017378    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"
Sep 10 19:34:59.021173 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.021105    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:34:59.022266 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.022240    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:34:59.025325 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.025300    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"]
Sep 10 19:34:59.086145 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.086098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:34:59.105181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.105134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88a69dbd-e6e9-402a-813f-de3995461d70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf\" (UID: \"88a69dbd-e6e9-402a-813f-de3995461d70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"
Sep 10 19:34:59.206367 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.206329    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88a69dbd-e6e9-402a-813f-de3995461d70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf\" (UID: \"88a69dbd-e6e9-402a-813f-de3995461d70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"
Sep 10 19:34:59.207072 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.207045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88a69dbd-e6e9-402a-813f-de3995461d70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf\" (UID: \"88a69dbd-e6e9-402a-813f-de3995461d70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"
Sep 10 19:34:59.331339 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.331236    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"
Sep 10 19:34:59.631111 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.631073    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"]
Sep 10 19:34:59.633189 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:34:59.633153    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a69dbd_e6e9_402a_813f_de3995461d70.slice/crio-ab7403d3e1e891a33c4d58df4ed8fea2ad6adfad8dfcb430eca9ff5c1dfdf1d5 WatchSource:0}: Error finding container ab7403d3e1e891a33c4d58df4ed8fea2ad6adfad8dfcb430eca9ff5c1dfdf1d5: Status 404 returned error can't find the container with id ab7403d3e1e891a33c4d58df4ed8fea2ad6adfad8dfcb430eca9ff5c1dfdf1d5
Sep 10 19:34:59.654175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.653695    2562 patch_prober.go:28] interesting pod/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 19:34:59.654175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.653756    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 19:34:59.969928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:34:59.969864    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf" event={"ID":"88a69dbd-e6e9-402a-813f-de3995461d70","Type":"ContainerStarted","Data":"ab7403d3e1e891a33c4d58df4ed8fea2ad6adfad8dfcb430eca9ff5c1dfdf1d5"}
Sep 10 19:35:00.046661 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:00.046615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:00.658184 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:00.658148    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:35:00.658607 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:00.658410    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:35:00.659114 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:00.659097    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:35:02.989120 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:02.989079    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf" event={"ID":"88a69dbd-e6e9-402a-813f-de3995461d70","Type":"ContainerStarted","Data":"c248f3b3eefed59cb2d3485ea71aab95703621868219c5677c9d69583a607ef3"}
Sep 10 19:35:03.006533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:03.006479    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf" podStartSLOduration=2.5394067160000002 podStartE2EDuration="5.006437758s" totalImagesPullingTime="2.467031042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:03.005338073 +0000 UTC m=+895.945371906" watchObservedRunningTime="2026-09-10 19:35:03.006437758 +0000 UTC m=+895.946471591"
Sep 10 19:35:03.603318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:03.603282    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:35:03.603504 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:03.603341    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:35:03.616154 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:03.616119    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:35:03.995240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:03.995203    2562 generic.go:356] "Generic (PLEG): container finished" podID="88a69dbd-e6e9-402a-813f-de3995461d70" containerID="c248f3b3eefed59cb2d3485ea71aab95703621868219c5677c9d69583a607ef3" exitCode=0
Sep 10 19:35:03.995641 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:03.995286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf" event={"ID":"88a69dbd-e6e9-402a-813f-de3995461d70","Type":"ContainerDied","Data":"c248f3b3eefed59cb2d3485ea71aab95703621868219c5677c9d69583a607ef3"}
Sep 10 19:35:04.006933 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:04.006882    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:35:04.936380 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:04.936342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:35:05.191626 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:05.191557    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"
Sep 10 19:35:05.272430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:05.272397    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88a69dbd-e6e9-402a-813f-de3995461d70-maas-api-service-ca\") pod \"88a69dbd-e6e9-402a-813f-de3995461d70\" (UID: \"88a69dbd-e6e9-402a-813f-de3995461d70\") "
Sep 10 19:35:05.272820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:05.272795    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88a69dbd-e6e9-402a-813f-de3995461d70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "88a69dbd-e6e9-402a-813f-de3995461d70" (UID: "88a69dbd-e6e9-402a-813f-de3995461d70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:35:05.373802 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:05.373761    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88a69dbd-e6e9-402a-813f-de3995461d70-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:35:06.006528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:06.006442    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"
Sep 10 19:35:06.006528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:06.006473    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf" event={"ID":"88a69dbd-e6e9-402a-813f-de3995461d70","Type":"ContainerDied","Data":"ab7403d3e1e891a33c4d58df4ed8fea2ad6adfad8dfcb430eca9ff5c1dfdf1d5"}
Sep 10 19:35:06.006528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:06.006523    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab7403d3e1e891a33c4d58df4ed8fea2ad6adfad8dfcb430eca9ff5c1dfdf1d5"
Sep 10 19:35:07.585321 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:07.585295    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:35:07.590357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:07.590333    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:35:07.697049 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:07.697014    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:35:07.741424 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:07.741394    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:35:07.747133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:07.747110    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:35:11.033699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:11.033661    2562 generic.go:356] "Generic (PLEG): container finished" podID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerID="5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c" exitCode=0
Sep 10 19:35:11.034142 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:11.033738    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" event={"ID":"0b863ade-c45e-45c0-8d00-d4cb4fff9546","Type":"ContainerDied","Data":"5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c"}
Sep 10 19:35:11.034203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:11.034144    2562 scope.go:117] "RemoveContainer" containerID="5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c"
Sep 10 19:35:11.035195 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:11.035163    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:35:11.035267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:11.035235    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:35:11.035304 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:11.035268    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:35:12.041789 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:12.041756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" event={"ID":"0b863ade-c45e-45c0-8d00-d4cb4fff9546","Type":"ContainerStarted","Data":"9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998"}
Sep 10 19:35:12.265141 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:12.265096    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:35:12.265386 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:12.265364    2562 patch_prober.go:28] interesting pod/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 10 19:35:12.265438 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:12.265416    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 10 19:35:13.264931 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:13.264864    2562 patch_prober.go:28] interesting pod/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 10 19:35:13.265345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:13.264955    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 10 19:35:13.540276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:13.540179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:14.265323 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:14.265280    2562 patch_prober.go:28] interesting pod/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 10 19:35:14.265960 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:14.265913    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 10 19:35:15.085146 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.085105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:15.175678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.175500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:15.265619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.265152    2562 patch_prober.go:28] interesting pod/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 10 19:35:15.265619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.265225    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 10 19:35:15.303197 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.303155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:15.403627 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.403574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:15.433438 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.433403    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"]
Sep 10 19:35:15.439859 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.439827    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7189cc9-bzsrf"]
Sep 10 19:35:15.503246 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.503198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:15.600270 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.600236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:15.697292 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.697204    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88a69dbd-e6e9-402a-813f-de3995461d70" path="/var/lib/kubelet/pods/88a69dbd-e6e9-402a-813f-de3995461d70/volumes"
Sep 10 19:35:15.984585 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:15.984447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:16.110831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:16.110775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:16.201549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:16.201510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:16.265550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:16.265429    2562 patch_prober.go:28] interesting pod/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 10 19:35:16.265550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:16.265521    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 10 19:35:16.325418 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:16.325385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:16.801542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:16.801503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:16.898653 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:16.898618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:17.265309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:17.265268    2562 patch_prober.go:28] interesting pod/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 10 19:35:17.265625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:17.265597    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 10 19:35:18.416493 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:18.416454    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:35:18.416934 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:18.416839    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:35:18.418049 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:18.418029    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:35:19.317879 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:19.317838    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:35:19.687793 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:19.687750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:19.801799 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:19.801763    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:19.961778 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:19.961686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:20.533838 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.533795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:20.628490 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.628457    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"]
Sep 10 19:35:20.628911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.628873    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a69dbd-e6e9-402a-813f-de3995461d70" containerName="revoke-keys"
Sep 10 19:35:20.628911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.628902    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a69dbd-e6e9-402a-813f-de3995461d70" containerName="revoke-keys"
Sep 10 19:35:20.629000 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.628990    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a69dbd-e6e9-402a-813f-de3995461d70" containerName="revoke-keys"
Sep 10 19:35:20.632173 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.632155    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.634937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.634908    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-fd056b79-dockercfg-mhzrq\""
Sep 10 19:35:20.635055 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.634942    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-fd056b79\""
Sep 10 19:35:20.641950 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.641922    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"]
Sep 10 19:35:20.651437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.651407    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"]
Sep 10 19:35:20.655357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.655333    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:20.667694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.667660    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"]
Sep 10 19:35:20.668950 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.668921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:20.727035 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.726993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgxqx\" (UniqueName: \"kubernetes.io/projected/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-kube-api-access-hgxqx\") pod \"payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.727035 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.727035    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.727467 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.727063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrn66\" (UniqueName: \"kubernetes.io/projected/a351555f-2b3e-4059-ba88-838709ce049e-kube-api-access-nrn66\") pod \"payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") " pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:20.727467 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.727222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a351555f-2b3e-4059-ba88-838709ce049e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") " pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:20.828020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.827922    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a351555f-2b3e-4059-ba88-838709ce049e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") " pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:20.828205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.828061    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgxqx\" (UniqueName: \"kubernetes.io/projected/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-kube-api-access-hgxqx\") pod \"payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.828205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.828094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.828205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.828137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrn66\" (UniqueName: \"kubernetes.io/projected/a351555f-2b3e-4059-ba88-838709ce049e-kube-api-access-nrn66\") pod \"payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") " pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:20.828798 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.828776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.828847 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.828781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a351555f-2b3e-4059-ba88-838709ce049e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") " pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:20.840215 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.840176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrn66\" (UniqueName: \"kubernetes.io/projected/a351555f-2b3e-4059-ba88-838709ce049e-kube-api-access-nrn66\") pod \"payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") " pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:20.840215 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.840204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgxqx\" (UniqueName: \"kubernetes.io/projected/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-kube-api-access-hgxqx\") pod \"payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.871813 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.871771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:20.944165 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.944136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:20.967363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:20.967324    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:21.222717 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:21.222690    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"]
Sep 10 19:35:21.251068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:21.250936    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"]
Sep 10 19:35:21.266181 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:35:21.266147    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda351555f_2b3e_4059_ba88_838709ce049e.slice/crio-98e7d6929d52b17da6ae168e5235f809e87d3b4a644403e48ec2e956c438ee8c WatchSource:0}: Error finding container 98e7d6929d52b17da6ae168e5235f809e87d3b4a644403e48ec2e956c438ee8c: Status 404 returned error can't find the container with id 98e7d6929d52b17da6ae168e5235f809e87d3b4a644403e48ec2e956c438ee8c
Sep 10 19:35:22.111251 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.111205    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh" event={"ID":"5a5ee036-be8a-4657-b6ad-83a6ea5976cf","Type":"ContainerStarted","Data":"e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519"}
Sep 10 19:35:22.111751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.111280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh" event={"ID":"5a5ee036-be8a-4657-b6ad-83a6ea5976cf","Type":"ContainerStarted","Data":"bdd1f9d077a87c956eb8766a748a57917019f8910740cc859575f6d02a561bc8"}
Sep 10 19:35:22.111751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.111361    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:22.112857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.112829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb" event={"ID":"a351555f-2b3e-4059-ba88-838709ce049e","Type":"ContainerStarted","Data":"e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b"}
Sep 10 19:35:22.112991 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.112870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb" event={"ID":"a351555f-2b3e-4059-ba88-838709ce049e","Type":"ContainerStarted","Data":"98e7d6929d52b17da6ae168e5235f809e87d3b4a644403e48ec2e956c438ee8c"}
Sep 10 19:35:22.112991 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.112922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:22.128693 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.128630    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh" podStartSLOduration=2.128608692 podStartE2EDuration="2.128608692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:22.128093578 +0000 UTC m=+915.068127413" watchObservedRunningTime="2026-09-10 19:35:22.128608692 +0000 UTC m=+915.068642526"
Sep 10 19:35:22.148082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:22.148027    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb" podStartSLOduration=2.148010065 podStartE2EDuration="2.148010065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:22.146534024 +0000 UTC m=+915.086567857" watchObservedRunningTime="2026-09-10 19:35:22.148010065 +0000 UTC m=+915.088043897"
Sep 10 19:35:32.798346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:32.798305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:33.123072 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:33.122981    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:33.123072 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:33.123065    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:40.001203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:40.001161    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:35:44.753818 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.753778    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"]
Sep 10 19:35:44.757505 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.757487    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"
Sep 10 19:35:44.760402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.760375    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:35:44.760504 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.760375    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:35:44.766435 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.766406    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"]
Sep 10 19:35:44.865097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.865060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg\" (UID: \"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"
Sep 10 19:35:44.966193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.966147    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg\" (UID: \"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"
Sep 10 19:35:44.967088 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:44.967060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg\" (UID: \"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"
Sep 10 19:35:45.003673 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:45.003631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:45.071053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:45.070964    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"
Sep 10 19:35:45.344272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:45.344236    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"]
Sep 10 19:35:45.346995 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:35:45.346960    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d34ed8b_fc1d_4c4e_85bf_2a00f0f5dca0.slice/crio-0f9f892f6c9cbc32c43f8bac60aae86b43615e0d875cbe6b2620f988600c86d6 WatchSource:0}: Error finding container 0f9f892f6c9cbc32c43f8bac60aae86b43615e0d875cbe6b2620f988600c86d6: Status 404 returned error can't find the container with id 0f9f892f6c9cbc32c43f8bac60aae86b43615e0d875cbe6b2620f988600c86d6
Sep 10 19:35:46.256331 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:46.256294    2562 generic.go:356] "Generic (PLEG): container finished" podID="5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0" containerID="fe25e12abc8cf2b0db7ff6cd9f177b06c16abee241d67ea47a609d2f601c5cd7" exitCode=0
Sep 10 19:35:46.256726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:46.256381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg" event={"ID":"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0","Type":"ContainerDied","Data":"fe25e12abc8cf2b0db7ff6cd9f177b06c16abee241d67ea47a609d2f601c5cd7"}
Sep 10 19:35:46.256726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:46.256432    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg" event={"ID":"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0","Type":"ContainerStarted","Data":"0f9f892f6c9cbc32c43f8bac60aae86b43615e0d875cbe6b2620f988600c86d6"}
Sep 10 19:35:47.263030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:47.262998    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:47.404263 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:47.404237    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"
Sep 10 19:35:47.490970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:47.490928    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0-maas-api-service-ca\") pod \"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0\" (UID: \"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0\") "
Sep 10 19:35:47.491278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:47.491250    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0" (UID: "5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:35:47.597969 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:47.594063    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:35:48.269424 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:48.269395    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"
Sep 10 19:35:48.269809 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:48.269394    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg" event={"ID":"5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0","Type":"ContainerDied","Data":"0f9f892f6c9cbc32c43f8bac60aae86b43615e0d875cbe6b2620f988600c86d6"}
Sep 10 19:35:48.269809 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:48.269517    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f9f892f6c9cbc32c43f8bac60aae86b43615e0d875cbe6b2620f988600c86d6"
Sep 10 19:35:51.323795 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.323757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:51.354983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.354943    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"]
Sep 10 19:35:51.355264 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.355232    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb" podUID="a351555f-2b3e-4059-ba88-838709ce049e" containerName="payload-processing" containerID="cri-o://e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b" gracePeriod=30
Sep 10 19:35:51.367828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.367793    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"]
Sep 10 19:35:51.368274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.368235    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh" podUID="5a5ee036-be8a-4657-b6ad-83a6ea5976cf" containerName="payload-pre-processing" containerID="cri-o://e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519" gracePeriod=30
Sep 10 19:35:51.398430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.398393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:51.462065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.462027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:51.521960 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.521922    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"]
Sep 10 19:35:51.527966 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.527930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:51.528168 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.528094    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fd056b79-rm2hg"]
Sep 10 19:35:51.625993 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.625965    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:51.650198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.649274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:51.662970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.661869    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:51.696437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.696401    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0" path="/var/lib/kubelet/pods/5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0/volumes"
Sep 10 19:35:51.733288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.733251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a351555f-2b3e-4059-ba88-838709ce049e-plugins-config-volume\") pod \"a351555f-2b3e-4059-ba88-838709ce049e\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") "
Sep 10 19:35:51.733466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.733315    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrn66\" (UniqueName: \"kubernetes.io/projected/a351555f-2b3e-4059-ba88-838709ce049e-kube-api-access-nrn66\") pod \"a351555f-2b3e-4059-ba88-838709ce049e\" (UID: \"a351555f-2b3e-4059-ba88-838709ce049e\") "
Sep 10 19:35:51.733765 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.733740    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a351555f-2b3e-4059-ba88-838709ce049e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a351555f-2b3e-4059-ba88-838709ce049e" (UID: "a351555f-2b3e-4059-ba88-838709ce049e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:35:51.735532 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.735511    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a351555f-2b3e-4059-ba88-838709ce049e-kube-api-access-nrn66" (OuterVolumeSpecName: "kube-api-access-nrn66") pod "a351555f-2b3e-4059-ba88-838709ce049e" (UID: "a351555f-2b3e-4059-ba88-838709ce049e"). InnerVolumeSpecName "kube-api-access-nrn66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:35:51.743263 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.743231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:51.834254 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.834152    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-plugins-config-volume\") pod \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") "
Sep 10 19:35:51.834437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.834281    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgxqx\" (UniqueName: \"kubernetes.io/projected/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-kube-api-access-hgxqx\") pod \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\" (UID: \"5a5ee036-be8a-4657-b6ad-83a6ea5976cf\") "
Sep 10 19:35:51.834554 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.834527    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a5ee036-be8a-4657-b6ad-83a6ea5976cf" (UID: "5a5ee036-be8a-4657-b6ad-83a6ea5976cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:35:51.834611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.834565    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a351555f-2b3e-4059-ba88-838709ce049e-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:35:51.834611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.834586    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrn66\" (UniqueName: \"kubernetes.io/projected/a351555f-2b3e-4059-ba88-838709ce049e-kube-api-access-nrn66\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:35:51.836397 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.836373    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-kube-api-access-hgxqx" (OuterVolumeSpecName: "kube-api-access-hgxqx") pod "5a5ee036-be8a-4657-b6ad-83a6ea5976cf" (UID: "5a5ee036-be8a-4657-b6ad-83a6ea5976cf"). InnerVolumeSpecName "kube-api-access-hgxqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:35:51.935190 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.935149    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:35:51.935190 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:51.935182    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgxqx\" (UniqueName: \"kubernetes.io/projected/5a5ee036-be8a-4657-b6ad-83a6ea5976cf-kube-api-access-hgxqx\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:35:52.137180 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.137088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:52.292981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.292943    2562 generic.go:356] "Generic (PLEG): container finished" podID="5a5ee036-be8a-4657-b6ad-83a6ea5976cf" containerID="e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519" exitCode=0
Sep 10 19:35:52.292981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.292982    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"
Sep 10 19:35:52.293216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.293026    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh" event={"ID":"5a5ee036-be8a-4657-b6ad-83a6ea5976cf","Type":"ContainerDied","Data":"e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519"}
Sep 10 19:35:52.293216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.293093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh" event={"ID":"5a5ee036-be8a-4657-b6ad-83a6ea5976cf","Type":"ContainerDied","Data":"bdd1f9d077a87c956eb8766a748a57917019f8910740cc859575f6d02a561bc8"}
Sep 10 19:35:52.293216 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.293131    2562 scope.go:117] "RemoveContainer" containerID="e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519"
Sep 10 19:35:52.294479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.294450    2562 generic.go:356] "Generic (PLEG): container finished" podID="a351555f-2b3e-4059-ba88-838709ce049e" containerID="e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b" exitCode=0
Sep 10 19:35:52.294585 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.294488    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb" event={"ID":"a351555f-2b3e-4059-ba88-838709ce049e","Type":"ContainerDied","Data":"e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b"}
Sep 10 19:35:52.294585 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.294570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb" event={"ID":"a351555f-2b3e-4059-ba88-838709ce049e","Type":"ContainerDied","Data":"98e7d6929d52b17da6ae168e5235f809e87d3b4a644403e48ec2e956c438ee8c"}
Sep 10 19:35:52.294682 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.294517    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"
Sep 10 19:35:52.302982 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.302962    2562 scope.go:117] "RemoveContainer" containerID="e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519"
Sep 10 19:35:52.303561 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:35:52.303537    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519\": container with ID starting with e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519 not found: ID does not exist" containerID="e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519"
Sep 10 19:35:52.303666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.303568    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519"} err="failed to get container status \"e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519\": rpc error: code = NotFound desc = could not find container \"e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519\": container with ID starting with e005b7f1132059c5885fcf2259944ece4e1b9abb2623867c28bc81415a14b519 not found: ID does not exist"
Sep 10 19:35:52.303666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.303587    2562 scope.go:117] "RemoveContainer" containerID="e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b"
Sep 10 19:35:52.312382 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.312362    2562 scope.go:117] "RemoveContainer" containerID="e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b"
Sep 10 19:35:52.312684 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:35:52.312662    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b\": container with ID starting with e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b not found: ID does not exist" containerID="e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b"
Sep 10 19:35:52.312764 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.312695    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b"} err="failed to get container status \"e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b\": rpc error: code = NotFound desc = could not find container \"e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b\": container with ID starting with e95d82162355ec862c1d91c3503e24cab54d9c5fc1d0cc140cc368a9b06d331b not found: ID does not exist"
Sep 10 19:35:52.322628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.322596    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"]
Sep 10 19:35:52.326146 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.326119    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fd056b79-5f48f85446-bh4jh"]
Sep 10 19:35:52.339714 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.339685    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"]
Sep 10 19:35:52.350605 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.350566    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fd056b79-57874c49b6-z5pnb"]
Sep 10 19:35:52.472453 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.472418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:52.589597 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:52.589563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:53.696022 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:53.695990    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a5ee036-be8a-4657-b6ad-83a6ea5976cf" path="/var/lib/kubelet/pods/5a5ee036-be8a-4657-b6ad-83a6ea5976cf/volumes"
Sep 10 19:35:53.696446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:53.696432    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a351555f-2b3e-4059-ba88-838709ce049e" path="/var/lib/kubelet/pods/a351555f-2b3e-4059-ba88-838709ce049e/volumes"
Sep 10 19:35:54.866690 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:54.866655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:54.962517 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:54.962481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:55.966345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966305    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"]
Sep 10 19:35:55.966734 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966723    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a351555f-2b3e-4059-ba88-838709ce049e" containerName="payload-processing"
Sep 10 19:35:55.966782 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966736    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a351555f-2b3e-4059-ba88-838709ce049e" containerName="payload-processing"
Sep 10 19:35:55.966782 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966765    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0" containerName="revoke-keys"
Sep 10 19:35:55.966782 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966772    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0" containerName="revoke-keys"
Sep 10 19:35:55.966782 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966780    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a5ee036-be8a-4657-b6ad-83a6ea5976cf" containerName="payload-pre-processing"
Sep 10 19:35:55.966925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966786    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5ee036-be8a-4657-b6ad-83a6ea5976cf" containerName="payload-pre-processing"
Sep 10 19:35:55.966925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966844    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a5ee036-be8a-4657-b6ad-83a6ea5976cf" containerName="payload-pre-processing"
Sep 10 19:35:55.966925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966855    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a351555f-2b3e-4059-ba88-838709ce049e" containerName="payload-processing"
Sep 10 19:35:55.966925 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.966863    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d34ed8b-fc1d-4c4e-85bf-2a00f0f5dca0" containerName="revoke-keys"
Sep 10 19:35:55.971233 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.971215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:55.978030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.977351    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a1590\"/\"openshift-service-ca.crt\""
Sep 10 19:35:55.978030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.977388    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a1590\"/\"test-model-3a1590-kserve-self-signed-certs\""
Sep 10 19:35:55.978030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.977670    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a1590\"/\"default-dockercfg-x96f7\""
Sep 10 19:35:55.978406 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.978382    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a1590\"/\"kube-root-ca.crt\""
Sep 10 19:35:55.981632 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:55.981607    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"]
Sep 10 19:35:56.027446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.027409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.074549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.074511    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-model-cache\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.074733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.074561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-dshm\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.074733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.074681    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tls-certs\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.074733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.074706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tmp-dir\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.074733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.074725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-home\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.074942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.074740    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcmpm\" (UniqueName: \"kubernetes.io/projected/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-kube-api-access-fcmpm\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.129230 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.129185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.175706 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.175666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-model-cache\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.175706 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.175711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-dshm\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.175978 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.175771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tls-certs\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.175978 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.175788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tmp-dir\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.175978 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.175803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-home\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.175978 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.175819    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcmpm\" (UniqueName: \"kubernetes.io/projected/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-kube-api-access-fcmpm\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.176191 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.176165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-model-cache\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.176253 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.176215    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tmp-dir\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.176329 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.176303    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-home\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.178190 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.178168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-dshm\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.178420 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.178403    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tls-certs\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.185294 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.185270    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcmpm\" (UniqueName: \"kubernetes.io/projected/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-kube-api-access-fcmpm\") pod \"test-model-3a1590-kserve-594c8459ff-9dmcr\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") " pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.277929 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.277779    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.282875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.282852    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:35:56.378816 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.378782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.571656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.571631    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"]
Sep 10 19:35:56.573485 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:35:56.573450    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcde16817_8855_4776_bd2c_d2d2b9eb4bb6.slice/crio-790f686c1268052ea4a9b9fd4cecd5fba1dcd167577bfda5148a973e6c07ca94 WatchSource:0}: Error finding container 790f686c1268052ea4a9b9fd4cecd5fba1dcd167577bfda5148a973e6c07ca94: Status 404 returned error can't find the container with id 790f686c1268052ea4a9b9fd4cecd5fba1dcd167577bfda5148a973e6c07ca94
Sep 10 19:35:56.857917 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.857799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:56.937789 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:56.937746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:35:57.034432 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.034397    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"]
Sep 10 19:35:57.038289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.038265    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.041044 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.041001    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b06cf0bb\""
Sep 10 19:35:57.041186 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.041083    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b06cf0bb-dockercfg-lwhxc\""
Sep 10 19:35:57.047655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.047625    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"]
Sep 10 19:35:57.186309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.186271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rzxx\" (UniqueName: \"kubernetes.io/projected/0b992491-3349-42c2-8c3b-b91a5112960d-kube-api-access-4rzxx\") pod \"payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.186513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.186482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b992491-3349-42c2-8c3b-b91a5112960d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.287740 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.287699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b992491-3349-42c2-8c3b-b91a5112960d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.287987 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.287807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rzxx\" (UniqueName: \"kubernetes.io/projected/0b992491-3349-42c2-8c3b-b91a5112960d-kube-api-access-4rzxx\") pod \"payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.288547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.288517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b992491-3349-42c2-8c3b-b91a5112960d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.296423 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.296399    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rzxx\" (UniqueName: \"kubernetes.io/projected/0b992491-3349-42c2-8c3b-b91a5112960d-kube-api-access-4rzxx\") pod \"payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.330656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.330621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr" event={"ID":"cde16817-8855-4776-bd2c-d2d2b9eb4bb6","Type":"ContainerStarted","Data":"0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f"}
Sep 10 19:35:57.330813 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.330665    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr" event={"ID":"cde16817-8855-4776-bd2c-d2d2b9eb4bb6","Type":"ContainerStarted","Data":"790f686c1268052ea4a9b9fd4cecd5fba1dcd167577bfda5148a973e6c07ca94"}
Sep 10 19:35:57.349852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.349794    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr" podStartSLOduration=2.349778388 podStartE2EDuration="2.349778388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:57.348187406 +0000 UTC m=+950.288221238" watchObservedRunningTime="2026-09-10 19:35:57.349778388 +0000 UTC m=+950.289812220"
Sep 10 19:35:57.352342 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.352316    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:35:57.626700 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:57.626666    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"]
Sep 10 19:35:57.628175 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:35:57.628147    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b992491_3349_42c2_8c3b_b91a5112960d.slice/crio-531e0e1288f04cd408c0d0b3e5ab3f3390e26ee6f129a1628aeb61e077b20bf7 WatchSource:0}: Error finding container 531e0e1288f04cd408c0d0b3e5ab3f3390e26ee6f129a1628aeb61e077b20bf7: Status 404 returned error can't find the container with id 531e0e1288f04cd408c0d0b3e5ab3f3390e26ee6f129a1628aeb61e077b20bf7
Sep 10 19:35:58.337859 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:58.337816    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg" event={"ID":"0b992491-3349-42c2-8c3b-b91a5112960d","Type":"ContainerStarted","Data":"1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529"}
Sep 10 19:35:58.338321 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:58.337912    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg" event={"ID":"0b992491-3349-42c2-8c3b-b91a5112960d","Type":"ContainerStarted","Data":"531e0e1288f04cd408c0d0b3e5ab3f3390e26ee6f129a1628aeb61e077b20bf7"}
Sep 10 19:35:58.361272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:58.361217    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg" podStartSLOduration=1.361199401 podStartE2EDuration="1.361199401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:58.359769663 +0000 UTC m=+951.299803496" watchObservedRunningTime="2026-09-10 19:35:58.361199401 +0000 UTC m=+951.301233271"
Sep 10 19:35:59.342774 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:35:59.342737    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:36:00.975629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:00.975590    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:01.082629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.082589    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"]
Sep 10 19:36:01.086221 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.086200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"
Sep 10 19:36:01.088858 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.088835    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:36:01.088981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.088929    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:36:01.093746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.093716    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"]
Sep 10 19:36:01.122231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.122189    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/620f63f3-d6b3-47e7-8f62-1f22f16bca50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8\" (UID: \"620f63f3-d6b3-47e7-8f62-1f22f16bca50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"
Sep 10 19:36:01.221138 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.221098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:01.223130 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.223101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/620f63f3-d6b3-47e7-8f62-1f22f16bca50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8\" (UID: \"620f63f3-d6b3-47e7-8f62-1f22f16bca50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"
Sep 10 19:36:01.223758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.223734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/620f63f3-d6b3-47e7-8f62-1f22f16bca50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8\" (UID: \"620f63f3-d6b3-47e7-8f62-1f22f16bca50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"
Sep 10 19:36:01.398414 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.398379    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"
Sep 10 19:36:01.669516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:01.669478    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"]
Sep 10 19:36:01.671696 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:36:01.671665    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620f63f3_d6b3_47e7_8f62_1f22f16bca50.slice/crio-5f40cb7779c46747e93cd92535ea1826e921388ae43e4133669a5ecc0ec17145 WatchSource:0}: Error finding container 5f40cb7779c46747e93cd92535ea1826e921388ae43e4133669a5ecc0ec17145: Status 404 returned error can't find the container with id 5f40cb7779c46747e93cd92535ea1826e921388ae43e4133669a5ecc0ec17145
Sep 10 19:36:02.359689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:02.359652    2562 generic.go:356] "Generic (PLEG): container finished" podID="620f63f3-d6b3-47e7-8f62-1f22f16bca50" containerID="8781762deba6f43c37dbe485dab55145c750784292fd5a5cd15d0a4e8f3dac27" exitCode=0
Sep 10 19:36:02.360185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:02.359741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8" event={"ID":"620f63f3-d6b3-47e7-8f62-1f22f16bca50","Type":"ContainerDied","Data":"8781762deba6f43c37dbe485dab55145c750784292fd5a5cd15d0a4e8f3dac27"}
Sep 10 19:36:02.360185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:02.359800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8" event={"ID":"620f63f3-d6b3-47e7-8f62-1f22f16bca50","Type":"ContainerStarted","Data":"5f40cb7779c46747e93cd92535ea1826e921388ae43e4133669a5ecc0ec17145"}
Sep 10 19:36:03.508071 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:03.508046    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"
Sep 10 19:36:03.546909 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:03.546850    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/620f63f3-d6b3-47e7-8f62-1f22f16bca50-maas-api-service-ca\") pod \"620f63f3-d6b3-47e7-8f62-1f22f16bca50\" (UID: \"620f63f3-d6b3-47e7-8f62-1f22f16bca50\") "
Sep 10 19:36:03.547410 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:03.547195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/620f63f3-d6b3-47e7-8f62-1f22f16bca50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "620f63f3-d6b3-47e7-8f62-1f22f16bca50" (UID: "620f63f3-d6b3-47e7-8f62-1f22f16bca50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:36:03.648369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:03.648329    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/620f63f3-d6b3-47e7-8f62-1f22f16bca50-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:04.373073 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:04.373037    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"
Sep 10 19:36:04.373073 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:04.373069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8" event={"ID":"620f63f3-d6b3-47e7-8f62-1f22f16bca50","Type":"ContainerDied","Data":"5f40cb7779c46747e93cd92535ea1826e921388ae43e4133669a5ecc0ec17145"}
Sep 10 19:36:04.373304 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:04.373122    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f40cb7779c46747e93cd92535ea1826e921388ae43e4133669a5ecc0ec17145"
Sep 10 19:36:04.503115 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:04.503077    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:36:04.794665 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:04.794626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:06.284033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:06.283997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:36:06.284445 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:06.284066    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:36:06.297247 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:06.297211    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:36:06.405054 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:06.405021    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:36:10.349961 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:10.349929    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:36:11.619524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.619481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:11.715832 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.715796    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"]
Sep 10 19:36:11.716084 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.716036    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt" podUID="2380d8f1-7459-4ebf-882b-3669411e7ac1" containerName="payload-processing" containerID="cri-o://950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce" gracePeriod=30
Sep 10 19:36:11.764450 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.764404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:11.865242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.865191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:11.919141 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.919104    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"]
Sep 10 19:36:11.921691 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.921662    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-18c3c3-dmbd8"]
Sep 10 19:36:11.932759 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.932678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:11.990942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:11.990910    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:36:12.039276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.039237    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqxw9\" (UniqueName: \"kubernetes.io/projected/2380d8f1-7459-4ebf-882b-3669411e7ac1-kube-api-access-cqxw9\") pod \"2380d8f1-7459-4ebf-882b-3669411e7ac1\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") "
Sep 10 19:36:12.039475 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.039347    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2380d8f1-7459-4ebf-882b-3669411e7ac1-plugins-config-volume\") pod \"2380d8f1-7459-4ebf-882b-3669411e7ac1\" (UID: \"2380d8f1-7459-4ebf-882b-3669411e7ac1\") "
Sep 10 19:36:12.039750 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.039714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2380d8f1-7459-4ebf-882b-3669411e7ac1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2380d8f1-7459-4ebf-882b-3669411e7ac1" (UID: "2380d8f1-7459-4ebf-882b-3669411e7ac1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:36:12.041591 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.041557    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2380d8f1-7459-4ebf-882b-3669411e7ac1-kube-api-access-cqxw9" (OuterVolumeSpecName: "kube-api-access-cqxw9") pod "2380d8f1-7459-4ebf-882b-3669411e7ac1" (UID: "2380d8f1-7459-4ebf-882b-3669411e7ac1"). InnerVolumeSpecName "kube-api-access-cqxw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:12.141174 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.141084    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2380d8f1-7459-4ebf-882b-3669411e7ac1-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:12.141174 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.141117    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqxw9\" (UniqueName: \"kubernetes.io/projected/2380d8f1-7459-4ebf-882b-3669411e7ac1-kube-api-access-cqxw9\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:12.298612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.298561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:12.427231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.427140    2562 generic.go:356] "Generic (PLEG): container finished" podID="2380d8f1-7459-4ebf-882b-3669411e7ac1" containerID="950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce" exitCode=0
Sep 10 19:36:12.427231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.427180    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"
Sep 10 19:36:12.427231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.427224    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt" event={"ID":"2380d8f1-7459-4ebf-882b-3669411e7ac1","Type":"ContainerDied","Data":"950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce"}
Sep 10 19:36:12.427461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.427271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt" event={"ID":"2380d8f1-7459-4ebf-882b-3669411e7ac1","Type":"ContainerDied","Data":"b0b979361b96cd2e085189c9934b0b734148ec7882fe92773b6c1b5d87f79bea"}
Sep 10 19:36:12.427461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.427295    2562 scope.go:117] "RemoveContainer" containerID="950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce"
Sep 10 19:36:12.437268 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.437248    2562 scope.go:117] "RemoveContainer" containerID="950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce"
Sep 10 19:36:12.437566 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:36:12.437545    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce\": container with ID starting with 950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce not found: ID does not exist" containerID="950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce"
Sep 10 19:36:12.437631 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.437574    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce"} err="failed to get container status \"950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce\": rpc error: code = NotFound desc = could not find container \"950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce\": container with ID starting with 950fe184c233605583e3a884ad965c0673292940ef29407958be4761ac1cf4ce not found: ID does not exist"
Sep 10 19:36:12.449577 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.449546    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"]
Sep 10 19:36:12.453340 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:12.453312    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-18c3c3-864cbc6c9f-8x2gt"]
Sep 10 19:36:13.696201 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:13.696169    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2380d8f1-7459-4ebf-882b-3669411e7ac1" path="/var/lib/kubelet/pods/2380d8f1-7459-4ebf-882b-3669411e7ac1/volumes"
Sep 10 19:36:13.696579 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:13.696516    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="620f63f3-d6b3-47e7-8f62-1f22f16bca50" path="/var/lib/kubelet/pods/620f63f3-d6b3-47e7-8f62-1f22f16bca50/volumes"
Sep 10 19:36:15.653770 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:15.653735    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:15.714749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:15.714683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:17.375408 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:17.375368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:18.709364 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:18.709325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:18.814721 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:18.814682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:18.998007 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:18.997909    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"]
Sep 10 19:36:18.998183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:18.998120    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy" containerID="cri-o://6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f" gracePeriod=30
Sep 10 19:36:19.020799 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:19.020755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:19.445962 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:19.445917    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:19.779184 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:19.779090    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:19.868479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:19.868440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:20.622767 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.622731    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"]
Sep 10 19:36:20.623153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.623140    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="620f63f3-d6b3-47e7-8f62-1f22f16bca50" containerName="revoke-keys"
Sep 10 19:36:20.623205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.623154    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="620f63f3-d6b3-47e7-8f62-1f22f16bca50" containerName="revoke-keys"
Sep 10 19:36:20.623205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.623169    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2380d8f1-7459-4ebf-882b-3669411e7ac1" containerName="payload-processing"
Sep 10 19:36:20.623205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.623175    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2380d8f1-7459-4ebf-882b-3669411e7ac1" containerName="payload-processing"
Sep 10 19:36:20.623347 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.623251    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="620f63f3-d6b3-47e7-8f62-1f22f16bca50" containerName="revoke-keys"
Sep 10 19:36:20.623347 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.623261    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2380d8f1-7459-4ebf-882b-3669411e7ac1" containerName="payload-processing"
Sep 10 19:36:20.627719 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.627697    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"
Sep 10 19:36:20.630435 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.630408    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:36:20.630556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.630487    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:36:20.634390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.634356    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"]
Sep 10 19:36:20.719980 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.719943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/871a60f3-007a-4236-ab37-d0f92df0a05a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml\" (UID: \"871a60f3-007a-4236-ab37-d0f92df0a05a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"
Sep 10 19:36:20.821315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.821269    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/871a60f3-007a-4236-ab37-d0f92df0a05a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml\" (UID: \"871a60f3-007a-4236-ab37-d0f92df0a05a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"
Sep 10 19:36:20.821916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.821862    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/871a60f3-007a-4236-ab37-d0f92df0a05a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml\" (UID: \"871a60f3-007a-4236-ab37-d0f92df0a05a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"
Sep 10 19:36:20.940991 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:20.940952    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"
Sep 10 19:36:21.217356 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:21.217328    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"]
Sep 10 19:36:21.219782 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:36:21.219753    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871a60f3_007a_4236_ab37_d0f92df0a05a.slice/crio-719ab3652d680e225925c697f0701fd3702baa6eb046bc76fa30d112d32c82d2 WatchSource:0}: Error finding container 719ab3652d680e225925c697f0701fd3702baa6eb046bc76fa30d112d32c82d2: Status 404 returned error can't find the container with id 719ab3652d680e225925c697f0701fd3702baa6eb046bc76fa30d112d32c82d2
Sep 10 19:36:21.483984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:21.483856    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml" event={"ID":"871a60f3-007a-4236-ab37-d0f92df0a05a","Type":"ContainerStarted","Data":"719ab3652d680e225925c697f0701fd3702baa6eb046bc76fa30d112d32c82d2"}
Sep 10 19:36:22.491752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:22.491708    2562 generic.go:356] "Generic (PLEG): container finished" podID="871a60f3-007a-4236-ab37-d0f92df0a05a" containerID="f3b731c03556b628c293cf07c095a46f9aafe70b5c55e3c507cbb4e32f6ec355" exitCode=0
Sep 10 19:36:22.492166 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:22.491753    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml" event={"ID":"871a60f3-007a-4236-ab37-d0f92df0a05a","Type":"ContainerDied","Data":"f3b731c03556b628c293cf07c095a46f9aafe70b5c55e3c507cbb4e32f6ec355"}
Sep 10 19:36:22.556732 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:22.556682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:22.647556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:22.647514    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:23.637376 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:23.637348    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"
Sep 10 19:36:23.752523 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:23.752425    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/871a60f3-007a-4236-ab37-d0f92df0a05a-maas-api-service-ca\") pod \"871a60f3-007a-4236-ab37-d0f92df0a05a\" (UID: \"871a60f3-007a-4236-ab37-d0f92df0a05a\") "
Sep 10 19:36:23.752816 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:23.752792    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/871a60f3-007a-4236-ab37-d0f92df0a05a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "871a60f3-007a-4236-ab37-d0f92df0a05a" (UID: "871a60f3-007a-4236-ab37-d0f92df0a05a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:36:23.839928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:23.839870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:23.853842 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:23.853805    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/871a60f3-007a-4236-ab37-d0f92df0a05a-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:23.945012 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:23.944945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:24.078075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.078034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:24.264504 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.264407    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:36:24.323931 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.323870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:24.359776 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.359736    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-envoy\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.359792    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-podinfo\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.359834    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-token\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.359929    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-socket\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.359969    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-certs\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.359997    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxhp2\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-kube-api-access-hxhp2\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360040    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-data\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360105    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istiod-ca-cert\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360146    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-credential-socket\") pod \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\" (UID: \"1a01fed9-04cd-47a1-837d-bf4d5c820bc8\") "
Sep 10 19:36:24.360289 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360242    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:24.360479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360283    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:24.360479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360446    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:24.360581 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360511    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.360581 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360528    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-workload-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.360705 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360668    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:24.360705 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.360675    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:36:24.362325 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.362303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:24.362421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.362323    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-kube-api-access-hxhp2" (OuterVolumeSpecName: "kube-api-access-hxhp2") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "kube-api-access-hxhp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:24.362499 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.362478    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:36:24.362537 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.362483    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a01fed9-04cd-47a1-837d-bf4d5c820bc8" (UID: "1a01fed9-04cd-47a1-837d-bf4d5c820bc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:24.412281 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.412231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:24.417079 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417049    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"]
Sep 10 19:36:24.417457 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417444    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy"
Sep 10 19:36:24.417506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417458    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy"
Sep 10 19:36:24.417506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417466    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="871a60f3-007a-4236-ab37-d0f92df0a05a" containerName="revoke-keys"
Sep 10 19:36:24.417506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417472    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="871a60f3-007a-4236-ab37-d0f92df0a05a" containerName="revoke-keys"
Sep 10 19:36:24.417506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417486    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy"
Sep 10 19:36:24.417506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417491    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy"
Sep 10 19:36:24.417657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417552    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy"
Sep 10 19:36:24.417657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417560    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="871a60f3-007a-4236-ab37-d0f92df0a05a" containerName="revoke-keys"
Sep 10 19:36:24.417657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.417570    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerName="istio-proxy"
Sep 10 19:36:24.420845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.420827    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.423376 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.423351    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-8f9c8a\""
Sep 10 19:36:24.423520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.423476    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-8f9c8a-dockercfg-qqrjr\""
Sep 10 19:36:24.430857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.430815    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"]
Sep 10 19:36:24.433379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.433346    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"]
Sep 10 19:36:24.436960 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.436936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:24.448491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.448462    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"]
Sep 10 19:36:24.461430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.461398    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istiod-ca-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.461430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.461426    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-credential-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.461430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.461436    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-envoy\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.461657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.461447    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-podinfo\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.461657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.461454    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-token\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.461657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.461462    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxhp2\" (UniqueName: \"kubernetes.io/projected/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-kube-api-access-hxhp2\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.461657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.461470    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a01fed9-04cd-47a1-837d-bf4d5c820bc8-istio-data\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:24.503959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.503928    2562 generic.go:356] "Generic (PLEG): container finished" podID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" containerID="6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f" exitCode=0
Sep 10 19:36:24.504121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.503988    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"
Sep 10 19:36:24.504121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.504001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" event={"ID":"1a01fed9-04cd-47a1-837d-bf4d5c820bc8","Type":"ContainerDied","Data":"6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f"}
Sep 10 19:36:24.504121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.504063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj" event={"ID":"1a01fed9-04cd-47a1-837d-bf4d5c820bc8","Type":"ContainerDied","Data":"f7c4e98c67208d4d07bc919deb431639d004504ed0f71999f3a1b81c06c3dec4"}
Sep 10 19:36:24.504121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.504096    2562 scope.go:117] "RemoveContainer" containerID="6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f"
Sep 10 19:36:24.505782 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.505760    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml" event={"ID":"871a60f3-007a-4236-ab37-d0f92df0a05a","Type":"ContainerDied","Data":"719ab3652d680e225925c697f0701fd3702baa6eb046bc76fa30d112d32c82d2"}
Sep 10 19:36:24.505908 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.505786    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"
Sep 10 19:36:24.505908 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.505805    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="719ab3652d680e225925c697f0701fd3702baa6eb046bc76fa30d112d32c82d2"
Sep 10 19:36:24.514600 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.514544    2562 scope.go:117] "RemoveContainer" containerID="fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786"
Sep 10 19:36:24.524141 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.524120    2562 scope.go:117] "RemoveContainer" containerID="6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f"
Sep 10 19:36:24.524461 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:36:24.524433    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f\": container with ID starting with 6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f not found: ID does not exist" containerID="6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f"
Sep 10 19:36:24.524524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.524475    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f"} err="failed to get container status \"6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f\": rpc error: code = NotFound desc = could not find container \"6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f\": container with ID starting with 6183f2ac65bc0ab18423417eee3cb2da239205b4b9a37b4ab65e5f3fefdeb02f not found: ID does not exist"
Sep 10 19:36:24.524524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.524499    2562 scope.go:117] "RemoveContainer" containerID="fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786"
Sep 10 19:36:24.524793 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:36:24.524774    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786\": container with ID starting with fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786 not found: ID does not exist" containerID="fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786"
Sep 10 19:36:24.524837 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.524798    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786"} err="failed to get container status \"fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786\": rpc error: code = NotFound desc = could not find container \"fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786\": container with ID starting with fc40a4df371b7e62be6c4bcf5f7a73a98867bfd6d2e8348d290b559f6e2d3786 not found: ID does not exist"
Sep 10 19:36:24.535354 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.535304    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"]
Sep 10 19:36:24.537338 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.537276    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-18c3c3-openshift-default-585476cfd4-7scsj"]
Sep 10 19:36:24.562835 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.562799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.563082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.562965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55227a81-0528-47be-9a79-555be7cf9a63-plugins-config-volume\") pod \"payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:24.563082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.563032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c57v9\" (UniqueName: \"kubernetes.io/projected/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-kube-api-access-c57v9\") pod \"payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.563082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.563060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6wql\" (UniqueName: \"kubernetes.io/projected/55227a81-0528-47be-9a79-555be7cf9a63-kube-api-access-n6wql\") pod \"payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:24.663630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.663576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.664092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.663680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55227a81-0528-47be-9a79-555be7cf9a63-plugins-config-volume\") pod \"payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:24.664092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.663721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c57v9\" (UniqueName: \"kubernetes.io/projected/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-kube-api-access-c57v9\") pod \"payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.664092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.663743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6wql\" (UniqueName: \"kubernetes.io/projected/55227a81-0528-47be-9a79-555be7cf9a63-kube-api-access-n6wql\") pod \"payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:24.664597 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.664571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.664638 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.664571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55227a81-0528-47be-9a79-555be7cf9a63-plugins-config-volume\") pod \"payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:24.672368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.672338    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6wql\" (UniqueName: \"kubernetes.io/projected/55227a81-0528-47be-9a79-555be7cf9a63-kube-api-access-n6wql\") pod \"payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") " pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:24.672593 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.672470    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c57v9\" (UniqueName: \"kubernetes.io/projected/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-kube-api-access-c57v9\") pod \"payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.734080 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.734038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:24.749368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:24.749341    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:25.012993 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.012966    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"]
Sep 10 19:36:25.014944 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:36:25.014909    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda906a3c4_fdbb_48d4_b51e_e4dfc9fc577d.slice/crio-a56c177e0bad2638aa6f09c27938153c3b11965aba4a4211d9716fc16e8d0a6e WatchSource:0}: Error finding container a56c177e0bad2638aa6f09c27938153c3b11965aba4a4211d9716fc16e8d0a6e: Status 404 returned error can't find the container with id a56c177e0bad2638aa6f09c27938153c3b11965aba4a4211d9716fc16e8d0a6e
Sep 10 19:36:25.034942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.034911    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"]
Sep 10 19:36:25.037176 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:36:25.037129    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55227a81_0528_47be_9a79_555be7cf9a63.slice/crio-1667ec9c8d26cafeb4a9ae098491922f5fe66a4aa765d3924c41f21ab143f0b6 WatchSource:0}: Error finding container 1667ec9c8d26cafeb4a9ae098491922f5fe66a4aa765d3924c41f21ab143f0b6: Status 404 returned error can't find the container with id 1667ec9c8d26cafeb4a9ae098491922f5fe66a4aa765d3924c41f21ab143f0b6
Sep 10 19:36:25.302769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.302672    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:36:25.513209 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.513168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld" event={"ID":"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d","Type":"ContainerStarted","Data":"e4a56a2fa16986be58a62eafb2d089d63cb2d454d68e8ff61c15153c79e9d5f3"}
Sep 10 19:36:25.513379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.513237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld" event={"ID":"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d","Type":"ContainerStarted","Data":"a56c177e0bad2638aa6f09c27938153c3b11965aba4a4211d9716fc16e8d0a6e"}
Sep 10 19:36:25.513379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.513266    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:25.516250 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.516227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj" event={"ID":"55227a81-0528-47be-9a79-555be7cf9a63","Type":"ContainerStarted","Data":"6a2d3134faf83f21ce45e01ea26e5acf0459a3cdc24ea243fa304e90100c097a"}
Sep 10 19:36:25.516399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.516268    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:25.516399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.516290    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj" event={"ID":"55227a81-0528-47be-9a79-555be7cf9a63","Type":"ContainerStarted","Data":"1667ec9c8d26cafeb4a9ae098491922f5fe66a4aa765d3924c41f21ab143f0b6"}
Sep 10 19:36:25.531553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.531496    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld" podStartSLOduration=1.531479815 podStartE2EDuration="1.531479815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:25.530875388 +0000 UTC m=+978.470909225" watchObservedRunningTime="2026-09-10 19:36:25.531479815 +0000 UTC m=+978.471513647"
Sep 10 19:36:25.546785 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.546727    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj" podStartSLOduration=1.546710139 podStartE2EDuration="1.546710139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:25.545316486 +0000 UTC m=+978.485350318" watchObservedRunningTime="2026-09-10 19:36:25.546710139 +0000 UTC m=+978.486743971"
Sep 10 19:36:25.696903 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:25.696865    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a01fed9-04cd-47a1-837d-bf4d5c820bc8" path="/var/lib/kubelet/pods/1a01fed9-04cd-47a1-837d-bf4d5c820bc8/volumes"
Sep 10 19:36:26.431008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:26.430968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:27.011383 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.011344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:27.040526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.040493    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"]
Sep 10 19:36:27.040781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.040747    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg" podUID="0b992491-3349-42c2-8c3b-b91a5112960d" containerName="payload-pre-processing" containerID="cri-o://1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529" gracePeriod=30
Sep 10 19:36:27.111422 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.111387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:27.198325 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.198289    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"]
Sep 10 19:36:27.202038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.202009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:27.202909 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.202872    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b06cf0bb-zlvml"]
Sep 10 19:36:27.262365 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.262253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:27.292826 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.292801    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:36:27.393996 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.393956    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rzxx\" (UniqueName: \"kubernetes.io/projected/0b992491-3349-42c2-8c3b-b91a5112960d-kube-api-access-4rzxx\") pod \"0b992491-3349-42c2-8c3b-b91a5112960d\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") "
Sep 10 19:36:27.394192 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.394005    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b992491-3349-42c2-8c3b-b91a5112960d-plugins-config-volume\") pod \"0b992491-3349-42c2-8c3b-b91a5112960d\" (UID: \"0b992491-3349-42c2-8c3b-b91a5112960d\") "
Sep 10 19:36:27.394378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.394354    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b992491-3349-42c2-8c3b-b91a5112960d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b992491-3349-42c2-8c3b-b91a5112960d" (UID: "0b992491-3349-42c2-8c3b-b91a5112960d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:36:27.396150 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.396128    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b992491-3349-42c2-8c3b-b91a5112960d-kube-api-access-4rzxx" (OuterVolumeSpecName: "kube-api-access-4rzxx") pod "0b992491-3349-42c2-8c3b-b91a5112960d" (UID: "0b992491-3349-42c2-8c3b-b91a5112960d"). InnerVolumeSpecName "kube-api-access-4rzxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:27.495377 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.495331    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rzxx\" (UniqueName: \"kubernetes.io/projected/0b992491-3349-42c2-8c3b-b91a5112960d-kube-api-access-4rzxx\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:27.495377 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.495367    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b992491-3349-42c2-8c3b-b91a5112960d-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:27.527837 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.527742    2562 generic.go:356] "Generic (PLEG): container finished" podID="0b992491-3349-42c2-8c3b-b91a5112960d" containerID="1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529" exitCode=0
Sep 10 19:36:27.527837 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.527787    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"
Sep 10 19:36:27.528067 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.527832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg" event={"ID":"0b992491-3349-42c2-8c3b-b91a5112960d","Type":"ContainerDied","Data":"1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529"}
Sep 10 19:36:27.528067 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.527900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg" event={"ID":"0b992491-3349-42c2-8c3b-b91a5112960d","Type":"ContainerDied","Data":"531e0e1288f04cd408c0d0b3e5ab3f3390e26ee6f129a1628aeb61e077b20bf7"}
Sep 10 19:36:27.528067 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.527913    2562 scope.go:117] "RemoveContainer" containerID="1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529"
Sep 10 19:36:27.537491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.537471    2562 scope.go:117] "RemoveContainer" containerID="1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529"
Sep 10 19:36:27.537811 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:36:27.537791    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529\": container with ID starting with 1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529 not found: ID does not exist" containerID="1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529"
Sep 10 19:36:27.537862 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.537821    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529"} err="failed to get container status \"1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529\": rpc error: code = NotFound desc = could not find container \"1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529\": container with ID starting with 1569ee0c951f49096e21e385cf5969b7f63ebd9f215fc4bd84d31a9594b99529 not found: ID does not exist"
Sep 10 19:36:27.549466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.549434    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"]
Sep 10 19:36:27.552752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.552715    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b06cf0bb-95c8fcd-sm2kg"]
Sep 10 19:36:27.702277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.696704    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b992491-3349-42c2-8c3b-b91a5112960d" path="/var/lib/kubelet/pods/0b992491-3349-42c2-8c3b-b91a5112960d/volumes"
Sep 10 19:36:27.702277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:27.697330    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="871a60f3-007a-4236-ab37-d0f92df0a05a" path="/var/lib/kubelet/pods/871a60f3-007a-4236-ab37-d0f92df0a05a/volumes"
Sep 10 19:36:28.388069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:28.388025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:28.824309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:28.824269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:29.159689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:29.159641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:29.575917 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:29.575809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:31.195854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:31.195821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:31.286263 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:31.286232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:32.361667 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.361629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:32.463288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.463250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:32.567458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.567413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:32.857413 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.857378    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:32.956641 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.956599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:32.988046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.988006    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"]
Sep 10 19:36:32.988451 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.988434    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b992491-3349-42c2-8c3b-b91a5112960d" containerName="payload-pre-processing"
Sep 10 19:36:32.988515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.988454    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b992491-3349-42c2-8c3b-b91a5112960d" containerName="payload-pre-processing"
Sep 10 19:36:32.988590 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.988576    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b992491-3349-42c2-8c3b-b91a5112960d" containerName="payload-pre-processing"
Sep 10 19:36:32.994846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.994823    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:32.997584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.997559    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-cdcbd691\""
Sep 10 19:36:32.997705 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:32.997600    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-cdcbd691-dockercfg-n8hf4\""
Sep 10 19:36:33.002843 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.002819    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"]
Sep 10 19:36:33.057060 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.057016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldl5q\" (UniqueName: \"kubernetes.io/projected/0e236c0a-d69f-46b4-be5e-6d332aaa167e-kube-api-access-ldl5q\") pod \"payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") " pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:33.057242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.057145    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e236c0a-d69f-46b4-be5e-6d332aaa167e-plugins-config-volume\") pod \"payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") " pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:33.158588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.158550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e236c0a-d69f-46b4-be5e-6d332aaa167e-plugins-config-volume\") pod \"payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") " pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:33.158741 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.158622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldl5q\" (UniqueName: \"kubernetes.io/projected/0e236c0a-d69f-46b4-be5e-6d332aaa167e-kube-api-access-ldl5q\") pod \"payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") " pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:33.159312 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.159286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e236c0a-d69f-46b4-be5e-6d332aaa167e-plugins-config-volume\") pod \"payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") " pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:33.166533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.166498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldl5q\" (UniqueName: \"kubernetes.io/projected/0e236c0a-d69f-46b4-be5e-6d332aaa167e-kube-api-access-ldl5q\") pod \"payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") " pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:33.306183 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.306141    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:33.580778 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:33.580750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"]
Sep 10 19:36:33.584115 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:36:33.584067    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e236c0a_d69f_46b4_be5e_6d332aaa167e.slice/crio-f2d90fd258c9437c4a1e9533c7f7c195668e00f1d848fcfb556f4220c9cff75b WatchSource:0}: Error finding container f2d90fd258c9437c4a1e9533c7f7c195668e00f1d848fcfb556f4220c9cff75b: Status 404 returned error can't find the container with id f2d90fd258c9437c4a1e9533c7f7c195668e00f1d848fcfb556f4220c9cff75b
Sep 10 19:36:34.578121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:34.578070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7" event={"ID":"0e236c0a-d69f-46b4-be5e-6d332aaa167e","Type":"ContainerStarted","Data":"0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d"}
Sep 10 19:36:34.578296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:34.578173    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7" event={"ID":"0e236c0a-d69f-46b4-be5e-6d332aaa167e","Type":"ContainerStarted","Data":"f2d90fd258c9437c4a1e9533c7f7c195668e00f1d848fcfb556f4220c9cff75b"}
Sep 10 19:36:34.578296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:34.578205    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:34.595793 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:34.595732    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7" podStartSLOduration=2.5957164 podStartE2EDuration="2.5957164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:34.594255546 +0000 UTC m=+987.534289378" watchObservedRunningTime="2026-09-10 19:36:34.5957164 +0000 UTC m=+987.535750327"
Sep 10 19:36:35.257913 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:35.257866    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"]
Sep 10 19:36:35.266715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:35.266678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:36.523562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:36.523525    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:36:36.523940 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:36.523706    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:36:44.715042 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:44.714999    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:36:44.878598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:44.878563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:45.586546 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:45.586508    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:36:47.513191 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:47.513141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:50.278672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.278619    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl" podUID="7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" containerName="main" containerID="cri-o://7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af" gracePeriod=45
Sep 10 19:36:50.536805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.536736    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:36:50.625659 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.625612    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-home\") pod \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") "
Sep 10 19:36:50.625834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.625678    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tls-certs\") pod \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") "
Sep 10 19:36:50.625834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.625722    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tmp-dir\") pod \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") "
Sep 10 19:36:50.625834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.625769    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-dshm\") pod \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") "
Sep 10 19:36:50.626029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.625936    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-model-cache\") pod \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") "
Sep 10 19:36:50.626029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.625948    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-home" (OuterVolumeSpecName: "home") pod "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" (UID: "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:50.626029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.625998    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmmbc\" (UniqueName: \"kubernetes.io/projected/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-kube-api-access-vmmbc\") pod \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\" (UID: \"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef\") "
Sep 10 19:36:50.626169 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.626063    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" (UID: "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:50.626217 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.626178    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" (UID: "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:50.626371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.626353    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:50.626442 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.626372    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:50.626442 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.626382    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:50.628187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.628158    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-kube-api-access-vmmbc" (OuterVolumeSpecName: "kube-api-access-vmmbc") pod "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" (UID: "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef"). InnerVolumeSpecName "kube-api-access-vmmbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:50.628301 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.628162    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" (UID: "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:36:50.628301 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.628218    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-dshm" (OuterVolumeSpecName: "dshm") pod "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" (UID: "7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:36:50.672731 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.672687    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" containerID="7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af" exitCode=0
Sep 10 19:36:50.672932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.672754    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"
Sep 10 19:36:50.672932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.672752    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl" event={"ID":"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef","Type":"ContainerDied","Data":"7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af"}
Sep 10 19:36:50.672932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.672867    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl" event={"ID":"7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef","Type":"ContainerDied","Data":"9b38d0255e09f0070a17f64437ec093d3324c2943661a90b2d13baa68e6e5163"}
Sep 10 19:36:50.672932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.672910    2562 scope.go:117] "RemoveContainer" containerID="7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af"
Sep 10 19:36:50.683149 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.683126    2562 scope.go:117] "RemoveContainer" containerID="7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af"
Sep 10 19:36:50.683486 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:36:50.683467    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af\": container with ID starting with 7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af not found: ID does not exist" containerID="7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af"
Sep 10 19:36:50.683531 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.683496    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af"} err="failed to get container status \"7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af\": rpc error: code = NotFound desc = could not find container \"7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af\": container with ID starting with 7a857a4bcf8fc906382ce97bf8b240b008558d0e58d32701ac851aa4a07602af not found: ID does not exist"
Sep 10 19:36:50.697331 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.697293    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"]
Sep 10 19:36:50.700530 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.700499    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-f4a65e71-kserve-74b5b4d9fb-jnlnl"]
Sep 10 19:36:50.727093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.727059    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:50.727093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.727089    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmmbc\" (UniqueName: \"kubernetes.io/projected/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-kube-api-access-vmmbc\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:50.727275 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:50.727099    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:36:51.696284 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:51.696245    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" path="/var/lib/kubelet/pods/7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef/volumes"
Sep 10 19:36:53.124318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.124283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:53.173451 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.173413    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"]
Sep 10 19:36:53.173904 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.173871    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" containerName="main"
Sep 10 19:36:53.174027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.173907    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" containerName="main"
Sep 10 19:36:53.174085 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.174047    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a17b45b-bf92-4c3e-ad30-e0d0f169a5ef" containerName="main"
Sep 10 19:36:53.178557 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.178531    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.182432 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.182405    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a037c8\"/\"test-model-a037c8-kserve-self-signed-certs\""
Sep 10 19:36:53.182586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.182435    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a037c8\"/\"default-dockercfg-2xlhx\""
Sep 10 19:36:53.182724 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.182703    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a037c8\"/\"kube-root-ca.crt\""
Sep 10 19:36:53.182845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.182759    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a037c8\"/\"openshift-service-ca.crt\""
Sep 10 19:36:53.186749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.186256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"]
Sep 10 19:36:53.251750 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.251716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-home\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.251956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.251801    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tmp-dir\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.251956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.251828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tls-certs\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.251956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.251872    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-model-cache\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.251956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.251943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr4vr\" (UniqueName: \"kubernetes.io/projected/86c2cfdb-2faf-4231-8e40-f8d2529f1002-kube-api-access-fr4vr\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.252125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.251970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-dshm\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.268677 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.268647    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:53.353085 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353042    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-model-cache\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353085 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr4vr\" (UniqueName: \"kubernetes.io/projected/86c2cfdb-2faf-4231-8e40-f8d2529f1002-kube-api-access-fr4vr\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-dshm\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353138    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-home\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353193    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tmp-dir\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tls-certs\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353559 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-model-cache\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353559 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353552    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-home\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.353639 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.353591    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tmp-dir\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.355568 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.355541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-dshm\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.355744 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.355727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tls-certs\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.361956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.361929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr4vr\" (UniqueName: \"kubernetes.io/projected/86c2cfdb-2faf-4231-8e40-f8d2529f1002-kube-api-access-fr4vr\") pod \"test-model-a037c8-kserve-696ff6595c-7ww54\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") " pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.489003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.488965    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:36:53.762715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.762691    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"]
Sep 10 19:36:53.765090 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:36:53.765056    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86c2cfdb_2faf_4231_8e40_f8d2529f1002.slice/crio-d831fc09a5175341d400ecaa05b135af31297a3a5c8434ef78764c732e7cdd56 WatchSource:0}: Error finding container d831fc09a5175341d400ecaa05b135af31297a3a5c8434ef78764c732e7cdd56: Status 404 returned error can't find the container with id d831fc09a5175341d400ecaa05b135af31297a3a5c8434ef78764c732e7cdd56
Sep 10 19:36:53.998983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:53.998942    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:36:54.697602 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:54.697555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54" event={"ID":"86c2cfdb-2faf-4231-8e40-f8d2529f1002","Type":"ContainerStarted","Data":"f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75"}
Sep 10 19:36:54.698045 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:54.697609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54" event={"ID":"86c2cfdb-2faf-4231-8e40-f8d2529f1002","Type":"ContainerStarted","Data":"d831fc09a5175341d400ecaa05b135af31297a3a5c8434ef78764c732e7cdd56"}
Sep 10 19:36:54.720491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:36:54.720437    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54" podStartSLOduration=1.720418062 podStartE2EDuration="1.720418062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:54.718713274 +0000 UTC m=+1007.658747107" watchObservedRunningTime="2026-09-10 19:36:54.720418062 +0000 UTC m=+1007.660451894"
Sep 10 19:37:03.489155 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:03.489108    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:37:03.489575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:03.489169    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:37:03.502575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:03.502546    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:37:03.758777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:03.758690    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:37:07.501796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:07.501751    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:37:11.358058 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.358017    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"]
Sep 10 19:37:11.363164 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.363138    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.367262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.367214    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-8f9c8a\"/\"explicit-model-8f9c8a-kserve-self-signed-certs\""
Sep 10 19:37:11.367262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.367253    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-8f9c8a\"/\"default-dockercfg-68b46\""
Sep 10 19:37:11.367489 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.367330    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-8f9c8a\"/\"openshift-service-ca.crt\""
Sep 10 19:37:11.367489 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.367223    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-8f9c8a\"/\"kube-root-ca.crt\""
Sep 10 19:37:11.372546 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.372522    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"]
Sep 10 19:37:11.394702 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.394665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:11.535746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.535701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-home\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.535746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.535744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9hcs\" (UniqueName: \"kubernetes.io/projected/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-kube-api-access-d9hcs\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.536074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.535766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tls-certs\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.536074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.535854    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-model-cache\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.536074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.535972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tmp-dir\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.536074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.536011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-dshm\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.636939 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.636801    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-dshm\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.636964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-home\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.636992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9hcs\" (UniqueName: \"kubernetes.io/projected/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-kube-api-access-d9hcs\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.637021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tls-certs\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.637052    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-model-cache\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.637100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tmp-dir\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637401 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.637364    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-home\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637459 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.637418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tmp-dir\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.637459 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.637447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-model-cache\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.639232 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.639211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-dshm\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.639421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.639399    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tls-certs\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.645724 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.645696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9hcs\" (UniqueName: \"kubernetes.io/projected/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-kube-api-access-d9hcs\") pod \"explicit-model-8f9c8a-kserve-748f5785c9-9vx2k\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") " pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.674338 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.674302    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:11.947268 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:11.947237    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"]
Sep 10 19:37:11.949273 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:37:11.949243    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7de5849_db21_4e4a_a1e8_32d1b55fda6f.slice/crio-3cf61a869b4f9daf7bfb6711bce885dd6ded9a5bbbf1315157cebbf550455642 WatchSource:0}: Error finding container 3cf61a869b4f9daf7bfb6711bce885dd6ded9a5bbbf1315157cebbf550455642: Status 404 returned error can't find the container with id 3cf61a869b4f9daf7bfb6711bce885dd6ded9a5bbbf1315157cebbf550455642
Sep 10 19:37:12.801226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:12.801169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k" event={"ID":"c7de5849-db21-4e4a-a1e8-32d1b55fda6f","Type":"ContainerStarted","Data":"47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453"}
Sep 10 19:37:12.801641 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:12.801240    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k" event={"ID":"c7de5849-db21-4e4a-a1e8-32d1b55fda6f","Type":"ContainerStarted","Data":"3cf61a869b4f9daf7bfb6711bce885dd6ded9a5bbbf1315157cebbf550455642"}
Sep 10 19:37:12.823373 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:12.823311    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k" podStartSLOduration=1.82329251 podStartE2EDuration="1.82329251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:37:12.820967257 +0000 UTC m=+1025.761001089" watchObservedRunningTime="2026-09-10 19:37:12.82329251 +0000 UTC m=+1025.763326342"
Sep 10 19:37:12.950952 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:12.950918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:12.990956 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:12.990917    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"]
Sep 10 19:37:12.991288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:12.991239    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7" podUID="0e236c0a-d69f-46b4-be5e-6d332aaa167e" containerName="payload-processing" containerID="cri-o://0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d" gracePeriod=30
Sep 10 19:37:13.050578 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.050541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:13.152986 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.152949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:13.250367 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.250332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:13.255434 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.255409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:37:13.334298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.334257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:13.353667 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.353627    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e236c0a-d69f-46b4-be5e-6d332aaa167e-plugins-config-volume\") pod \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") "
Sep 10 19:37:13.353834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.353677    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldl5q\" (UniqueName: \"kubernetes.io/projected/0e236c0a-d69f-46b4-be5e-6d332aaa167e-kube-api-access-ldl5q\") pod \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\" (UID: \"0e236c0a-d69f-46b4-be5e-6d332aaa167e\") "
Sep 10 19:37:13.354093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.354060    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e236c0a-d69f-46b4-be5e-6d332aaa167e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e236c0a-d69f-46b4-be5e-6d332aaa167e" (UID: "0e236c0a-d69f-46b4-be5e-6d332aaa167e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:37:13.355803 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.355779    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e236c0a-d69f-46b4-be5e-6d332aaa167e-kube-api-access-ldl5q" (OuterVolumeSpecName: "kube-api-access-ldl5q") pod "0e236c0a-d69f-46b4-be5e-6d332aaa167e" (UID: "0e236c0a-d69f-46b4-be5e-6d332aaa167e"). InnerVolumeSpecName "kube-api-access-ldl5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:37:13.455146 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.455097    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e236c0a-d69f-46b4-be5e-6d332aaa167e-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:13.455146 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.455136    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldl5q\" (UniqueName: \"kubernetes.io/projected/0e236c0a-d69f-46b4-be5e-6d332aaa167e-kube-api-access-ldl5q\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:13.807875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.807840    2562 generic.go:356] "Generic (PLEG): container finished" podID="0e236c0a-d69f-46b4-be5e-6d332aaa167e" containerID="0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d" exitCode=0
Sep 10 19:37:13.808344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.807916    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"
Sep 10 19:37:13.808344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.807923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7" event={"ID":"0e236c0a-d69f-46b4-be5e-6d332aaa167e","Type":"ContainerDied","Data":"0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d"}
Sep 10 19:37:13.808344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.807984    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7" event={"ID":"0e236c0a-d69f-46b4-be5e-6d332aaa167e","Type":"ContainerDied","Data":"f2d90fd258c9437c4a1e9533c7f7c195668e00f1d848fcfb556f4220c9cff75b"}
Sep 10 19:37:13.808344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.808007    2562 scope.go:117] "RemoveContainer" containerID="0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d"
Sep 10 19:37:13.819914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.819870    2562 scope.go:117] "RemoveContainer" containerID="0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d"
Sep 10 19:37:13.820251 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:37:13.820227    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d\": container with ID starting with 0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d not found: ID does not exist" containerID="0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d"
Sep 10 19:37:13.820316 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.820260    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d"} err="failed to get container status \"0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d\": rpc error: code = NotFound desc = could not find container \"0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d\": container with ID starting with 0be988d2a0c82e38e97627ca6deca8d568a48729cee7e933671e3702b416bc9d not found: ID does not exist"
Sep 10 19:37:13.825685 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.825656    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"]
Sep 10 19:37:13.831226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.831198    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cdcbd691-6795565745-2r2g7"]
Sep 10 19:37:13.999837 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:13.999744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:14.092356 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:14.092317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:14.194465 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:14.194420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:15.629242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:15.629190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:15.696179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:15.696148    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e236c0a-d69f-46b4-be5e-6d332aaa167e" path="/var/lib/kubelet/pods/0e236c0a-d69f-46b4-be5e-6d332aaa167e/volumes"
Sep 10 19:37:20.702731 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:20.702688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:21.339423 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.339382    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:21.448718 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.448680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:21.540718 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.540680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:21.640825 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.640794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:21.674648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.674608    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:21.674841 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.674673    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:21.687401 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.687366    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:21.862601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:21.862566    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:22.046355 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:22.046272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:28.004494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:28.004455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:37:34.130565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:34.130528    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"]
Sep 10 19:37:34.156464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:34.156415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:34.372561 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:34.372527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:43.996433 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:43.996390    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:37:44.744151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.744110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:44.776574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.776533    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"]
Sep 10 19:37:44.776879 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.776849    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj" podUID="55227a81-0528-47be-9a79-555be7cf9a63" containerName="payload-processing" containerID="cri-o://6a2d3134faf83f21ce45e01ea26e5acf0459a3cdc24ea243fa304e90100c097a" gracePeriod=30
Sep 10 19:37:44.782267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.782176    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"]
Sep 10 19:37:44.782762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.782713    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld" podUID="a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" containerName="payload-pre-processing" containerID="cri-o://e4a56a2fa16986be58a62eafb2d089d63cb2d454d68e8ff61c15153c79e9d5f3" gracePeriod=30
Sep 10 19:37:44.829744 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.829702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:44.890302 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.890261    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:44.953346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.953150    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:44.972868 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.972830    2562 generic.go:356] "Generic (PLEG): container finished" podID="a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" containerID="e4a56a2fa16986be58a62eafb2d089d63cb2d454d68e8ff61c15153c79e9d5f3" exitCode=0
Sep 10 19:37:44.973057 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.972911    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld" event={"ID":"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d","Type":"ContainerDied","Data":"e4a56a2fa16986be58a62eafb2d089d63cb2d454d68e8ff61c15153c79e9d5f3"}
Sep 10 19:37:44.974612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.974584    2562 generic.go:356] "Generic (PLEG): container finished" podID="55227a81-0528-47be-9a79-555be7cf9a63" containerID="6a2d3134faf83f21ce45e01ea26e5acf0459a3cdc24ea243fa304e90100c097a" exitCode=0
Sep 10 19:37:44.974746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:44.974652    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj" event={"ID":"55227a81-0528-47be-9a79-555be7cf9a63","Type":"ContainerDied","Data":"6a2d3134faf83f21ce45e01ea26e5acf0459a3cdc24ea243fa304e90100c097a"}
Sep 10 19:37:45.068135 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.068106    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:37:45.071329 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.071305    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:37:45.179868 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.179830    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55227a81-0528-47be-9a79-555be7cf9a63-plugins-config-volume\") pod \"55227a81-0528-47be-9a79-555be7cf9a63\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") "
Sep 10 19:37:45.179868 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.179873    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c57v9\" (UniqueName: \"kubernetes.io/projected/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-kube-api-access-c57v9\") pod \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") "
Sep 10 19:37:45.180165 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.179931    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-plugins-config-volume\") pod \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\" (UID: \"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d\") "
Sep 10 19:37:45.180165 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.180044    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6wql\" (UniqueName: \"kubernetes.io/projected/55227a81-0528-47be-9a79-555be7cf9a63-kube-api-access-n6wql\") pod \"55227a81-0528-47be-9a79-555be7cf9a63\" (UID: \"55227a81-0528-47be-9a79-555be7cf9a63\") "
Sep 10 19:37:45.180361 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.180336    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55227a81-0528-47be-9a79-555be7cf9a63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55227a81-0528-47be-9a79-555be7cf9a63" (UID: "55227a81-0528-47be-9a79-555be7cf9a63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:37:45.180421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.180391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" (UID: "a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:37:45.182147 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.182125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-kube-api-access-c57v9" (OuterVolumeSpecName: "kube-api-access-c57v9") pod "a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" (UID: "a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d"). InnerVolumeSpecName "kube-api-access-c57v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:37:45.182228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.182157    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55227a81-0528-47be-9a79-555be7cf9a63-kube-api-access-n6wql" (OuterVolumeSpecName: "kube-api-access-n6wql") pod "55227a81-0528-47be-9a79-555be7cf9a63" (UID: "55227a81-0528-47be-9a79-555be7cf9a63"). InnerVolumeSpecName "kube-api-access-n6wql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:37:45.281726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.281677    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6wql\" (UniqueName: \"kubernetes.io/projected/55227a81-0528-47be-9a79-555be7cf9a63-kube-api-access-n6wql\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:45.281726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.281719    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55227a81-0528-47be-9a79-555be7cf9a63-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:45.281726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.281729    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c57v9\" (UniqueName: \"kubernetes.io/projected/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-kube-api-access-c57v9\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:45.281726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.281737    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:45.300305 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.300219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:45.821284 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.821238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:45.921506 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.921472    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:45.981222 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.981134    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"
Sep 10 19:37:45.981222 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.981177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj" event={"ID":"55227a81-0528-47be-9a79-555be7cf9a63","Type":"ContainerDied","Data":"1667ec9c8d26cafeb4a9ae098491922f5fe66a4aa765d3924c41f21ab143f0b6"}
Sep 10 19:37:45.981447 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.981247    2562 scope.go:117] "RemoveContainer" containerID="6a2d3134faf83f21ce45e01ea26e5acf0459a3cdc24ea243fa304e90100c097a"
Sep 10 19:37:45.982750 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.982725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld" event={"ID":"a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d","Type":"ContainerDied","Data":"a56c177e0bad2638aa6f09c27938153c3b11965aba4a4211d9716fc16e8d0a6e"}
Sep 10 19:37:45.982857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.982766    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"
Sep 10 19:37:45.990723 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:45.990701    2562 scope.go:117] "RemoveContainer" containerID="e4a56a2fa16986be58a62eafb2d089d63cb2d454d68e8ff61c15153c79e9d5f3"
Sep 10 19:37:46.006603 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:46.006571    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"]
Sep 10 19:37:46.012363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:46.012333    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-8f9c8a-549785d985-dpgkj"]
Sep 10 19:37:46.023235 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:46.023201    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"]
Sep 10 19:37:46.027505 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:46.027478    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-8f9c8a-5587c9f7d6-8pbld"]
Sep 10 19:37:47.696601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:47.696559    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55227a81-0528-47be-9a79-555be7cf9a63" path="/var/lib/kubelet/pods/55227a81-0528-47be-9a79-555be7cf9a63/volumes"
Sep 10 19:37:47.697172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:47.697148    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" path="/var/lib/kubelet/pods/a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d/volumes"
Sep 10 19:37:47.988834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:47.988749    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:49.151302 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.151262    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k" podUID="c7de5849-db21-4e4a-a1e8-32d1b55fda6f" containerName="main" containerID="cri-o://47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453" gracePeriod=45
Sep 10 19:37:49.405614 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.405547    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:49.420574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.420532    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-dshm\") pod \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") "
Sep 10 19:37:49.420574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.420582    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-home\") pod \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") "
Sep 10 19:37:49.420834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.420614    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9hcs\" (UniqueName: \"kubernetes.io/projected/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-kube-api-access-d9hcs\") pod \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") "
Sep 10 19:37:49.420834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.420650    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tmp-dir\") pod \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") "
Sep 10 19:37:49.420834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.420722    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tls-certs\") pod \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") "
Sep 10 19:37:49.420834 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.420770    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-model-cache\") pod \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\" (UID: \"c7de5849-db21-4e4a-a1e8-32d1b55fda6f\") "
Sep 10 19:37:49.421077 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.420949    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-home" (OuterVolumeSpecName: "home") pod "c7de5849-db21-4e4a-a1e8-32d1b55fda6f" (UID: "c7de5849-db21-4e4a-a1e8-32d1b55fda6f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:49.421077 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.421057    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c7de5849-db21-4e4a-a1e8-32d1b55fda6f" (UID: "c7de5849-db21-4e4a-a1e8-32d1b55fda6f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:49.421179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.421104    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7de5849-db21-4e4a-a1e8-32d1b55fda6f" (UID: "c7de5849-db21-4e4a-a1e8-32d1b55fda6f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:49.421245 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.421191    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:49.421245 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.421204    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:49.421245 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.421216    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:49.422987 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.422958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-dshm" (OuterVolumeSpecName: "dshm") pod "c7de5849-db21-4e4a-a1e8-32d1b55fda6f" (UID: "c7de5849-db21-4e4a-a1e8-32d1b55fda6f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:49.423131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.423021    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7de5849-db21-4e4a-a1e8-32d1b55fda6f" (UID: "c7de5849-db21-4e4a-a1e8-32d1b55fda6f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:37:49.423131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.423103    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-kube-api-access-d9hcs" (OuterVolumeSpecName: "kube-api-access-d9hcs") pod "c7de5849-db21-4e4a-a1e8-32d1b55fda6f" (UID: "c7de5849-db21-4e4a-a1e8-32d1b55fda6f"). InnerVolumeSpecName "kube-api-access-d9hcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:37:49.522628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.522590    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:49.522628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.522624    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9hcs\" (UniqueName: \"kubernetes.io/projected/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-kube-api-access-d9hcs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:49.522628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:49.522635    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7de5849-db21-4e4a-a1e8-32d1b55fda6f-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:37:50.008739 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.008627    2562 generic.go:356] "Generic (PLEG): container finished" podID="c7de5849-db21-4e4a-a1e8-32d1b55fda6f" containerID="47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453" exitCode=0
Sep 10 19:37:50.008739 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.008667    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"
Sep 10 19:37:50.008739 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.008724    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k" event={"ID":"c7de5849-db21-4e4a-a1e8-32d1b55fda6f","Type":"ContainerDied","Data":"47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453"}
Sep 10 19:37:50.009020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.008770    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k" event={"ID":"c7de5849-db21-4e4a-a1e8-32d1b55fda6f","Type":"ContainerDied","Data":"3cf61a869b4f9daf7bfb6711bce885dd6ded9a5bbbf1315157cebbf550455642"}
Sep 10 19:37:50.009020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.008789    2562 scope.go:117] "RemoveContainer" containerID="47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453"
Sep 10 19:37:50.017864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.017843    2562 scope.go:117] "RemoveContainer" containerID="47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453"
Sep 10 19:37:50.018206 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:37:50.018184    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453\": container with ID starting with 47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453 not found: ID does not exist" containerID="47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453"
Sep 10 19:37:50.018280 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.018215    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453"} err="failed to get container status \"47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453\": rpc error: code = NotFound desc = could not find container \"47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453\": container with ID starting with 47c6c8de5cde004b31365996fdc2dabf7eba7275f5db2ef66ff77856bc4d5453 not found: ID does not exist"
Sep 10 19:37:50.027064 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.027028    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"]
Sep 10 19:37:50.030314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:50.030288    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-8f9c8a/explicit-model-8f9c8a-kserve-748f5785c9-9vx2k"]
Sep 10 19:37:51.363430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:51.363386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:51.436753 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:51.436712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:51.610749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:51.610713    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:51.696984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:51.696950    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7de5849-db21-4e4a-a1e8-32d1b55fda6f" path="/var/lib/kubelet/pods/c7de5849-db21-4e4a-a1e8-32d1b55fda6f/volumes"
Sep 10 19:37:52.035197 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:52.035105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:52.346401 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:52.346317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:52.631231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:52.631117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:53.484639 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:53.484599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:53.648738 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:53.648688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:53.811963 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:53.811857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:54.797373 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:54.797337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:54.902963 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:54.902925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:55.008532 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:55.008491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:37:59.792856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:37:59.792820    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:38:01.854447 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854410    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"]
Sep 10 19:38:01.854840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854783    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" containerName="payload-pre-processing"
Sep 10 19:38:01.854840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854794    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" containerName="payload-pre-processing"
Sep 10 19:38:01.854840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854802    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7de5849-db21-4e4a-a1e8-32d1b55fda6f" containerName="main"
Sep 10 19:38:01.854840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854808    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7de5849-db21-4e4a-a1e8-32d1b55fda6f" containerName="main"
Sep 10 19:38:01.854840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854828    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55227a81-0528-47be-9a79-555be7cf9a63" containerName="payload-processing"
Sep 10 19:38:01.854840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854834    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="55227a81-0528-47be-9a79-555be7cf9a63" containerName="payload-processing"
Sep 10 19:38:01.854840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854842    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e236c0a-d69f-46b4-be5e-6d332aaa167e" containerName="payload-processing"
Sep 10 19:38:01.855092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854847    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e236c0a-d69f-46b4-be5e-6d332aaa167e" containerName="payload-processing"
Sep 10 19:38:01.855092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854923    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7de5849-db21-4e4a-a1e8-32d1b55fda6f" containerName="main"
Sep 10 19:38:01.855092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854930    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="55227a81-0528-47be-9a79-555be7cf9a63" containerName="payload-processing"
Sep 10 19:38:01.855092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854939    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a906a3c4-fdbb-48d4-b51e-e4dfc9fc577d" containerName="payload-pre-processing"
Sep 10 19:38:01.855092 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.854946    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e236c0a-d69f-46b4-be5e-6d332aaa167e" containerName="payload-processing"
Sep 10 19:38:01.859521 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.859494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:01.863319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.863286    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-9947fa\"/\"openshift-service-ca.crt\""
Sep 10 19:38:01.863319 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.863304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-9947fa\"/\"noten-model-9947fa-kserve-self-signed-certs\""
Sep 10 19:38:01.863507 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.863360    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-9947fa\"/\"kube-root-ca.crt\""
Sep 10 19:38:01.863507 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.863294    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-9947fa\"/\"default-dockercfg-2pgjc\""
Sep 10 19:38:01.871029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.871002    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"]
Sep 10 19:38:01.945322 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.945280    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-dshm\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:01.945483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.945333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tmp-dir\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:01.945483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.945371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-home\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:01.945483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.945390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvfh9\" (UniqueName: \"kubernetes.io/projected/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-kube-api-access-rvfh9\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:01.945483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.945429    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tls-certs\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:01.945657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.945506    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-model-cache\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:01.979170 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:01.979132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:02.046151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvfh9\" (UniqueName: \"kubernetes.io/projected/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-kube-api-access-rvfh9\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.046323 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tls-certs\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.046323 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-model-cache\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.046323 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-dshm\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.046491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tmp-dir\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.046491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-home\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.046781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-model-cache\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.046781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046771    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-home\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.047022 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.046814    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tmp-dir\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.048605 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.048581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-dshm\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.048762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.048745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tls-certs\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.054249 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.054226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvfh9\" (UniqueName: \"kubernetes.io/projected/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-kube-api-access-rvfh9\") pod \"noten-model-9947fa-kserve-86cdc756c9-jrx2t\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") " pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.169305 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.169258    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:02.445192 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:02.445126    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"]
Sep 10 19:38:02.447543 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:38:02.447514    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod704983e9_b7bc_4fd9_8efd_dfc8f3e90153.slice/crio-5041f46f6b397ef424fcd6fff9b337a34e8a097bea8aa18211feb4d0c64b9157 WatchSource:0}: Error finding container 5041f46f6b397ef424fcd6fff9b337a34e8a097bea8aa18211feb4d0c64b9157: Status 404 returned error can't find the container with id 5041f46f6b397ef424fcd6fff9b337a34e8a097bea8aa18211feb4d0c64b9157
Sep 10 19:38:03.080497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:03.080454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t" event={"ID":"704983e9-b7bc-4fd9-8efd-dfc8f3e90153","Type":"ContainerStarted","Data":"2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317"}
Sep 10 19:38:03.080497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:03.080500    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t" event={"ID":"704983e9-b7bc-4fd9-8efd-dfc8f3e90153","Type":"ContainerStarted","Data":"5041f46f6b397ef424fcd6fff9b337a34e8a097bea8aa18211feb4d0c64b9157"}
Sep 10 19:38:03.097292 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:03.097237    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t" podStartSLOduration=2.097222271 podStartE2EDuration="2.097222271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:38:03.095879558 +0000 UTC m=+1076.035913391" watchObservedRunningTime="2026-09-10 19:38:03.097222271 +0000 UTC m=+1076.037256103"
Sep 10 19:38:10.856628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:10.856591    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"]
Sep 10 19:38:10.876599 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:10.876560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:11.048111 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:11.048068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:11.119268 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:11.119173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:11.219612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:11.219574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:11.327537 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:11.327490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:12.169461 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:12.169429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:16.403986 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:16.403949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:17.201464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:17.201419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:17.293930 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:17.293874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:25.879877 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:25.879813    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t" podUID="704983e9-b7bc-4fd9-8efd-dfc8f3e90153" containerName="main" containerID="cri-o://2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317" gracePeriod=45
Sep 10 19:38:26.137957 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.137875    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:26.202067 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.202025    2562 generic.go:356] "Generic (PLEG): container finished" podID="704983e9-b7bc-4fd9-8efd-dfc8f3e90153" containerID="2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317" exitCode=0
Sep 10 19:38:26.202274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.202087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t" event={"ID":"704983e9-b7bc-4fd9-8efd-dfc8f3e90153","Type":"ContainerDied","Data":"2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317"}
Sep 10 19:38:26.202274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.202092    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"
Sep 10 19:38:26.202274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.202120    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t" event={"ID":"704983e9-b7bc-4fd9-8efd-dfc8f3e90153","Type":"ContainerDied","Data":"5041f46f6b397ef424fcd6fff9b337a34e8a097bea8aa18211feb4d0c64b9157"}
Sep 10 19:38:26.202274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.202137    2562 scope.go:117] "RemoveContainer" containerID="2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317"
Sep 10 19:38:26.210896 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.210863    2562 scope.go:117] "RemoveContainer" containerID="2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317"
Sep 10 19:38:26.211222 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:38:26.211200    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317\": container with ID starting with 2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317 not found: ID does not exist" containerID="2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317"
Sep 10 19:38:26.211305 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.211233    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317"} err="failed to get container status \"2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317\": rpc error: code = NotFound desc = could not find container \"2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317\": container with ID starting with 2e738803a5e0640cafe604308c63f22ce52b653702c86b9c58b7e5168d6f4317 not found: ID does not exist"
Sep 10 19:38:26.288919 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.288870    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-model-cache\") pod \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") "
Sep 10 19:38:26.289125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.288929    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-dshm\") pod \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") "
Sep 10 19:38:26.289125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.288956    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tls-certs\") pod \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") "
Sep 10 19:38:26.289125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289014    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tmp-dir\") pod \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") "
Sep 10 19:38:26.289125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289038    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvfh9\" (UniqueName: \"kubernetes.io/projected/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-kube-api-access-rvfh9\") pod \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") "
Sep 10 19:38:26.289125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289084    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-home\") pod \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\" (UID: \"704983e9-b7bc-4fd9-8efd-dfc8f3e90153\") "
Sep 10 19:38:26.289430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-model-cache" (OuterVolumeSpecName: "model-cache") pod "704983e9-b7bc-4fd9-8efd-dfc8f3e90153" (UID: "704983e9-b7bc-4fd9-8efd-dfc8f3e90153"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:26.289430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289314    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "704983e9-b7bc-4fd9-8efd-dfc8f3e90153" (UID: "704983e9-b7bc-4fd9-8efd-dfc8f3e90153"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:26.289534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289442    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-home" (OuterVolumeSpecName: "home") pod "704983e9-b7bc-4fd9-8efd-dfc8f3e90153" (UID: "704983e9-b7bc-4fd9-8efd-dfc8f3e90153"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:26.289534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289499    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:26.289611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.289541    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:26.291296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.291265    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "704983e9-b7bc-4fd9-8efd-dfc8f3e90153" (UID: "704983e9-b7bc-4fd9-8efd-dfc8f3e90153"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:38:26.291584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.291565    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-dshm" (OuterVolumeSpecName: "dshm") pod "704983e9-b7bc-4fd9-8efd-dfc8f3e90153" (UID: "704983e9-b7bc-4fd9-8efd-dfc8f3e90153"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:26.291652 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.291568    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-kube-api-access-rvfh9" (OuterVolumeSpecName: "kube-api-access-rvfh9") pod "704983e9-b7bc-4fd9-8efd-dfc8f3e90153" (UID: "704983e9-b7bc-4fd9-8efd-dfc8f3e90153"). InnerVolumeSpecName "kube-api-access-rvfh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:38:26.390379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.390344    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:26.390379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.390379    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:26.390582 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.390391    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvfh9\" (UniqueName: \"kubernetes.io/projected/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-kube-api-access-rvfh9\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:26.390582 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.390401    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/704983e9-b7bc-4fd9-8efd-dfc8f3e90153-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:26.523399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.523363    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"]
Sep 10 19:38:26.527873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:26.527842    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-9947fa/noten-model-9947fa-kserve-86cdc756c9-jrx2t"]
Sep 10 19:38:27.695600 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:27.695560    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="704983e9-b7bc-4fd9-8efd-dfc8f3e90153" path="/var/lib/kubelet/pods/704983e9-b7bc-4fd9-8efd-dfc8f3e90153/volumes"
Sep 10 19:38:27.798021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:27.797983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:28.262972 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:28.262934    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"]
Sep 10 19:38:28.286275 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:28.286245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:28.589033 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:28.588933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:29.258873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.258833    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"]
Sep 10 19:38:29.284617 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.284581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:29.391895 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.391851    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"]
Sep 10 19:38:29.392288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.392271    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704983e9-b7bc-4fd9-8efd-dfc8f3e90153" containerName="main"
Sep 10 19:38:29.392332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.392290    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="704983e9-b7bc-4fd9-8efd-dfc8f3e90153" containerName="main"
Sep 10 19:38:29.392386 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.392376    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="704983e9-b7bc-4fd9-8efd-dfc8f3e90153" containerName="main"
Sep 10 19:38:29.396789 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.396771    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"
Sep 10 19:38:29.399776 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.399749    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:38:29.399915 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.399825    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:38:29.402847 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.402820    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"]
Sep 10 19:38:29.518258 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.518166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/178fdd43-16be-464e-9b28-c9f1988463e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz\" (UID: \"178fdd43-16be-464e-9b28-c9f1988463e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"
Sep 10 19:38:29.533768 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.533724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:29.619553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.619509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/178fdd43-16be-464e-9b28-c9f1988463e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz\" (UID: \"178fdd43-16be-464e-9b28-c9f1988463e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"
Sep 10 19:38:29.620285 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.620263    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/178fdd43-16be-464e-9b28-c9f1988463e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz\" (UID: \"178fdd43-16be-464e-9b28-c9f1988463e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"
Sep 10 19:38:29.709276 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.709233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"
Sep 10 19:38:29.980807 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:29.980779    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"]
Sep 10 19:38:29.982590 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:38:29.982562    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178fdd43_16be_464e_9b28_c9f1988463e9.slice/crio-6db86ec38a04d73438ee5c6d25256982ff6e6c66b287e25603b1e04273559909 WatchSource:0}: Error finding container 6db86ec38a04d73438ee5c6d25256982ff6e6c66b287e25603b1e04273559909: Status 404 returned error can't find the container with id 6db86ec38a04d73438ee5c6d25256982ff6e6c66b287e25603b1e04273559909
Sep 10 19:38:30.226366 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:30.226327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz" event={"ID":"178fdd43-16be-464e-9b28-c9f1988463e9","Type":"ContainerStarted","Data":"6db86ec38a04d73438ee5c6d25256982ff6e6c66b287e25603b1e04273559909"}
Sep 10 19:38:31.234169 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:31.234129    2562 generic.go:356] "Generic (PLEG): container finished" podID="178fdd43-16be-464e-9b28-c9f1988463e9" containerID="6a6a2ad4f774b9a17e9883a20d3fd28aff0236dc981f06f37714408723246e45" exitCode=0
Sep 10 19:38:31.234563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:31.234214    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz" event={"ID":"178fdd43-16be-464e-9b28-c9f1988463e9","Type":"ContainerDied","Data":"6a6a2ad4f774b9a17e9883a20d3fd28aff0236dc981f06f37714408723246e45"}
Sep 10 19:38:32.309231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:32.309198    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:38:32.374988 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:32.374961    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"
Sep 10 19:38:32.442236 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:32.442196    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/178fdd43-16be-464e-9b28-c9f1988463e9-maas-api-service-ca\") pod \"178fdd43-16be-464e-9b28-c9f1988463e9\" (UID: \"178fdd43-16be-464e-9b28-c9f1988463e9\") "
Sep 10 19:38:32.442601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:32.442574    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178fdd43-16be-464e-9b28-c9f1988463e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "178fdd43-16be-464e-9b28-c9f1988463e9" (UID: "178fdd43-16be-464e-9b28-c9f1988463e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:38:32.543473 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:32.543377    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/178fdd43-16be-464e-9b28-c9f1988463e9-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:33.245876 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:33.245846    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"
Sep 10 19:38:33.246073 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:33.245879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz" event={"ID":"178fdd43-16be-464e-9b28-c9f1988463e9","Type":"ContainerDied","Data":"6db86ec38a04d73438ee5c6d25256982ff6e6c66b287e25603b1e04273559909"}
Sep 10 19:38:33.246073 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:33.245948    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6db86ec38a04d73438ee5c6d25256982ff6e6c66b287e25603b1e04273559909"
Sep 10 19:38:34.556273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:34.556234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:34.664616 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:34.664578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:34.757808 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:34.757767    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:34.854962 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:34.854858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:35.477937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:35.477873    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:35.574437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:35.574398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:35.674549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:35.674511    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:37.183298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:37.183264    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:39.874588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:39.874543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:39.949102 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:39.949070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:40.014491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:40.014453    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:40.071387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:40.071350    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"]
Sep 10 19:38:40.074087 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:40.074059    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a1590-wrnqz"]
Sep 10 19:38:40.158847 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:40.158815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:40.439327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:40.439233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:41.695782 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:41.695740    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="178fdd43-16be-464e-9b28-c9f1988463e9" path="/var/lib/kubelet/pods/178fdd43-16be-464e-9b28-c9f1988463e9/volumes"
Sep 10 19:38:43.283200 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.283155    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr" podUID="cde16817-8855-4776-bd2c-d2d2b9eb4bb6" containerName="main" containerID="cri-o://0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f" gracePeriod=45
Sep 10 19:38:43.437593 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.437523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:43.533567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.533494    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:38:43.537363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.537310    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:43.641113 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641080    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tmp-dir\") pod \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") "
Sep 10 19:38:43.641318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641145    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcmpm\" (UniqueName: \"kubernetes.io/projected/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-kube-api-access-fcmpm\") pod \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") "
Sep 10 19:38:43.641318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641163    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-model-cache\") pod \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") "
Sep 10 19:38:43.641318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641206    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tls-certs\") pod \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") "
Sep 10 19:38:43.641318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641235    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-home\") pod \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") "
Sep 10 19:38:43.641318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641304    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-dshm\") pod \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\" (UID: \"cde16817-8855-4776-bd2c-d2d2b9eb4bb6\") "
Sep 10 19:38:43.641569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641381    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cde16817-8855-4776-bd2c-d2d2b9eb4bb6" (UID: "cde16817-8855-4776-bd2c-d2d2b9eb4bb6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:43.641569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641474    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-model-cache" (OuterVolumeSpecName: "model-cache") pod "cde16817-8855-4776-bd2c-d2d2b9eb4bb6" (UID: "cde16817-8855-4776-bd2c-d2d2b9eb4bb6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:43.641569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641512    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-home" (OuterVolumeSpecName: "home") pod "cde16817-8855-4776-bd2c-d2d2b9eb4bb6" (UID: "cde16817-8855-4776-bd2c-d2d2b9eb4bb6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:43.641707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641613    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:43.641707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641627    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:43.641707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.641637    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:43.643494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.643459    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-dshm" (OuterVolumeSpecName: "dshm") pod "cde16817-8855-4776-bd2c-d2d2b9eb4bb6" (UID: "cde16817-8855-4776-bd2c-d2d2b9eb4bb6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:43.643494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.643478    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cde16817-8855-4776-bd2c-d2d2b9eb4bb6" (UID: "cde16817-8855-4776-bd2c-d2d2b9eb4bb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:38:43.643642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.643556    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-kube-api-access-fcmpm" (OuterVolumeSpecName: "kube-api-access-fcmpm") pod "cde16817-8855-4776-bd2c-d2d2b9eb4bb6" (UID: "cde16817-8855-4776-bd2c-d2d2b9eb4bb6"). InnerVolumeSpecName "kube-api-access-fcmpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:38:43.742553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.742509    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:43.742553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.742549    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcmpm\" (UniqueName: \"kubernetes.io/projected/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-kube-api-access-fcmpm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:43.742553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:43.742563    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cde16817-8855-4776-bd2c-d2d2b9eb4bb6-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:44.097411 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.097377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:44.283111 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.283041    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54" podUID="86c2cfdb-2faf-4231-8e40-f8d2529f1002" containerName="main" containerID="cri-o://f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75" gracePeriod=45
Sep 10 19:38:44.305129 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.305097    2562 generic.go:356] "Generic (PLEG): container finished" podID="cde16817-8855-4776-bd2c-d2d2b9eb4bb6" containerID="0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f" exitCode=0
Sep 10 19:38:44.305563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.305144    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"
Sep 10 19:38:44.305563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.305183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr" event={"ID":"cde16817-8855-4776-bd2c-d2d2b9eb4bb6","Type":"ContainerDied","Data":"0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f"}
Sep 10 19:38:44.305563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.305229    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr" event={"ID":"cde16817-8855-4776-bd2c-d2d2b9eb4bb6","Type":"ContainerDied","Data":"790f686c1268052ea4a9b9fd4cecd5fba1dcd167577bfda5148a973e6c07ca94"}
Sep 10 19:38:44.305563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.305250    2562 scope.go:117] "RemoveContainer" containerID="0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f"
Sep 10 19:38:44.351848 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.351757    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"]
Sep 10 19:38:44.355628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.355599    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-3a1590/test-model-3a1590-kserve-594c8459ff-9dmcr"]
Sep 10 19:38:44.394518 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.394488    2562 scope.go:117] "RemoveContainer" containerID="0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f"
Sep 10 19:38:44.394918 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:38:44.394871    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f\": container with ID starting with 0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f not found: ID does not exist" containerID="0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f"
Sep 10 19:38:44.395009 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.394929    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f"} err="failed to get container status \"0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f\": rpc error: code = NotFound desc = could not find container \"0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f\": container with ID starting with 0b252e39f2ae6bf37bf59ab72626f6ccd2cd57d02e1029d3af0205b56f8c367f not found: ID does not exist"
Sep 10 19:38:44.530674 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.530644    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:38:44.551050 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551017    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-model-cache\") pod \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") "
Sep 10 19:38:44.551232 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551074    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr4vr\" (UniqueName: \"kubernetes.io/projected/86c2cfdb-2faf-4231-8e40-f8d2529f1002-kube-api-access-fr4vr\") pod \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") "
Sep 10 19:38:44.551232 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551145    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-dshm\") pod \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") "
Sep 10 19:38:44.551232 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551183    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tls-certs\") pod \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") "
Sep 10 19:38:44.551232 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551210    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-home\") pod \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") "
Sep 10 19:38:44.551463 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551241    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tmp-dir\") pod \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\" (UID: \"86c2cfdb-2faf-4231-8e40-f8d2529f1002\") "
Sep 10 19:38:44.551463 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551343    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-model-cache" (OuterVolumeSpecName: "model-cache") pod "86c2cfdb-2faf-4231-8e40-f8d2529f1002" (UID: "86c2cfdb-2faf-4231-8e40-f8d2529f1002"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:44.551761 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551571    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "86c2cfdb-2faf-4231-8e40-f8d2529f1002" (UID: "86c2cfdb-2faf-4231-8e40-f8d2529f1002"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:44.551761 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.551575    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:44.552051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.552017    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-home" (OuterVolumeSpecName: "home") pod "86c2cfdb-2faf-4231-8e40-f8d2529f1002" (UID: "86c2cfdb-2faf-4231-8e40-f8d2529f1002"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:44.554080 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.554054    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "86c2cfdb-2faf-4231-8e40-f8d2529f1002" (UID: "86c2cfdb-2faf-4231-8e40-f8d2529f1002"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:38:44.554200 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.554073    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c2cfdb-2faf-4231-8e40-f8d2529f1002-kube-api-access-fr4vr" (OuterVolumeSpecName: "kube-api-access-fr4vr") pod "86c2cfdb-2faf-4231-8e40-f8d2529f1002" (UID: "86c2cfdb-2faf-4231-8e40-f8d2529f1002"). InnerVolumeSpecName "kube-api-access-fr4vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:38:44.554481 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.554458    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-dshm" (OuterVolumeSpecName: "dshm") pod "86c2cfdb-2faf-4231-8e40-f8d2529f1002" (UID: "86c2cfdb-2faf-4231-8e40-f8d2529f1002"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:44.652267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.652233    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr4vr\" (UniqueName: \"kubernetes.io/projected/86c2cfdb-2faf-4231-8e40-f8d2529f1002-kube-api-access-fr4vr\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:44.652267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.652265    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:44.652267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.652275    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:44.652505 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.652287    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:44.652505 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.652299    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c2cfdb-2faf-4231-8e40-f8d2529f1002-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:44.703441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:44.703405    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:38:45.312569 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.312535    2562 generic.go:356] "Generic (PLEG): container finished" podID="86c2cfdb-2faf-4231-8e40-f8d2529f1002" containerID="f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75" exitCode=0
Sep 10 19:38:45.313105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.312597    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"
Sep 10 19:38:45.313105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.312611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54" event={"ID":"86c2cfdb-2faf-4231-8e40-f8d2529f1002","Type":"ContainerDied","Data":"f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75"}
Sep 10 19:38:45.313105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.312642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54" event={"ID":"86c2cfdb-2faf-4231-8e40-f8d2529f1002","Type":"ContainerDied","Data":"d831fc09a5175341d400ecaa05b135af31297a3a5c8434ef78764c732e7cdd56"}
Sep 10 19:38:45.313105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.312660    2562 scope.go:117] "RemoveContainer" containerID="f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75"
Sep 10 19:38:45.321814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.321790    2562 scope.go:117] "RemoveContainer" containerID="f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75"
Sep 10 19:38:45.322140 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:38:45.322119    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75\": container with ID starting with f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75 not found: ID does not exist" containerID="f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75"
Sep 10 19:38:45.322192 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.322150    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75"} err="failed to get container status \"f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75\": rpc error: code = NotFound desc = could not find container \"f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75\": container with ID starting with f9b777523b741b4ad2fddc53abf70317f69236e58ef07096b7cbf04307466b75 not found: ID does not exist"
Sep 10 19:38:45.346839 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.346803    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"]
Sep 10 19:38:45.355293 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.355261    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a037c8/test-model-a037c8-kserve-696ff6595c-7ww54"]
Sep 10 19:38:45.702854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.702806    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86c2cfdb-2faf-4231-8e40-f8d2529f1002" path="/var/lib/kubelet/pods/86c2cfdb-2faf-4231-8e40-f8d2529f1002/volumes"
Sep 10 19:38:45.703410 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:45.703391    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde16817-8855-4776-bd2c-d2d2b9eb4bb6" path="/var/lib/kubelet/pods/cde16817-8855-4776-bd2c-d2d2b9eb4bb6/volumes"
Sep 10 19:38:46.735733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:46.735695    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:46.836045 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:46.836007    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:46.982632 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:46.982601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:47.346994 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.346949    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"]
Sep 10 19:38:47.347369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347355    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="178fdd43-16be-464e-9b28-c9f1988463e9" containerName="revoke-keys"
Sep 10 19:38:47.347415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347372    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="178fdd43-16be-464e-9b28-c9f1988463e9" containerName="revoke-keys"
Sep 10 19:38:47.347415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347391    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde16817-8855-4776-bd2c-d2d2b9eb4bb6" containerName="main"
Sep 10 19:38:47.347415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347397    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde16817-8855-4776-bd2c-d2d2b9eb4bb6" containerName="main"
Sep 10 19:38:47.347415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347403    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c2cfdb-2faf-4231-8e40-f8d2529f1002" containerName="main"
Sep 10 19:38:47.347415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347409    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c2cfdb-2faf-4231-8e40-f8d2529f1002" containerName="main"
Sep 10 19:38:47.347574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347468    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="178fdd43-16be-464e-9b28-c9f1988463e9" containerName="revoke-keys"
Sep 10 19:38:47.347574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347479    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde16817-8855-4776-bd2c-d2d2b9eb4bb6" containerName="main"
Sep 10 19:38:47.347574 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.347485    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c2cfdb-2faf-4231-8e40-f8d2529f1002" containerName="main"
Sep 10 19:38:47.352239 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.352215    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"
Sep 10 19:38:47.354801 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.354780    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:38:47.354942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.354871    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:38:47.359503 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.359474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"]
Sep 10 19:38:47.452083 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.452042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:47.479565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.479522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8100a9-8ea1-40b0-986a-6fbff712a6f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r\" (UID: \"ad8100a9-8ea1-40b0-986a-6fbff712a6f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"
Sep 10 19:38:47.580743 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.580695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8100a9-8ea1-40b0-986a-6fbff712a6f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r\" (UID: \"ad8100a9-8ea1-40b0-986a-6fbff712a6f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"
Sep 10 19:38:47.581379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.581359    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8100a9-8ea1-40b0-986a-6fbff712a6f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r\" (UID: \"ad8100a9-8ea1-40b0-986a-6fbff712a6f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"
Sep 10 19:38:47.665528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.665495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"
Sep 10 19:38:47.941133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:47.941101    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"]
Sep 10 19:38:47.943072 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:38:47.943045    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8100a9_8ea1_40b0_986a_6fbff712a6f5.slice/crio-304d6420ee5bcd1331e3db614f37e8c725dbd8733882e8e233ad9187fcfacb49 WatchSource:0}: Error finding container 304d6420ee5bcd1331e3db614f37e8c725dbd8733882e8e233ad9187fcfacb49: Status 404 returned error can't find the container with id 304d6420ee5bcd1331e3db614f37e8c725dbd8733882e8e233ad9187fcfacb49
Sep 10 19:38:48.329749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:48.329638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r" event={"ID":"ad8100a9-8ea1-40b0-986a-6fbff712a6f5","Type":"ContainerStarted","Data":"8edcea87d63e0ffd65a426cd4354bbde38e9593e579d806a5e73e8432227616c"}
Sep 10 19:38:48.329749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:48.329694    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r" event={"ID":"ad8100a9-8ea1-40b0-986a-6fbff712a6f5","Type":"ContainerStarted","Data":"304d6420ee5bcd1331e3db614f37e8c725dbd8733882e8e233ad9187fcfacb49"}
Sep 10 19:38:48.346777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:48.346714    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r" podStartSLOduration=1.34668959 podStartE2EDuration="1.34668959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:38:48.345309951 +0000 UTC m=+1121.285343784" watchObservedRunningTime="2026-09-10 19:38:48.34668959 +0000 UTC m=+1121.286723424"
Sep 10 19:38:49.336330 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:49.336288    2562 generic.go:356] "Generic (PLEG): container finished" podID="ad8100a9-8ea1-40b0-986a-6fbff712a6f5" containerID="8edcea87d63e0ffd65a426cd4354bbde38e9593e579d806a5e73e8432227616c" exitCode=0
Sep 10 19:38:49.336751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:49.336383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r" event={"ID":"ad8100a9-8ea1-40b0-986a-6fbff712a6f5","Type":"ContainerDied","Data":"8edcea87d63e0ffd65a426cd4354bbde38e9593e579d806a5e73e8432227616c"}
Sep 10 19:38:50.084357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.084326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:50.246563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.246527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:50.345823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.345790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:50.477085 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.477054    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"
Sep 10 19:38:50.609902 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.609787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8100a9-8ea1-40b0-986a-6fbff712a6f5-maas-api-service-ca\") pod \"ad8100a9-8ea1-40b0-986a-6fbff712a6f5\" (UID: \"ad8100a9-8ea1-40b0-986a-6fbff712a6f5\") "
Sep 10 19:38:50.610153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.610127    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad8100a9-8ea1-40b0-986a-6fbff712a6f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad8100a9-8ea1-40b0-986a-6fbff712a6f5" (UID: "ad8100a9-8ea1-40b0-986a-6fbff712a6f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:38:50.614093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.614058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:50.710762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.710721    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad8100a9-8ea1-40b0-986a-6fbff712a6f5-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:50.849178 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:50.849140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:51.348756 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:51.348715    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"
Sep 10 19:38:51.349269 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:51.348716    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r" event={"ID":"ad8100a9-8ea1-40b0-986a-6fbff712a6f5","Type":"ContainerDied","Data":"304d6420ee5bcd1331e3db614f37e8c725dbd8733882e8e233ad9187fcfacb49"}
Sep 10 19:38:51.349269 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:51.348854    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="304d6420ee5bcd1331e3db614f37e8c725dbd8733882e8e233ad9187fcfacb49"
Sep 10 19:38:53.413028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:53.412995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:53.509312 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:53.509278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:54.457831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:54.457786    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:38:54.756628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:54.756536    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:54.831113 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:54.831074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:54.974845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:54.974810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:55.241560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.241523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:55.353969 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.353932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:55.362633 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.362604    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"]
Sep 10 19:38:55.363040 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.363025    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad8100a9-8ea1-40b0-986a-6fbff712a6f5" containerName="revoke-keys"
Sep 10 19:38:55.363089 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.363042    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad8100a9-8ea1-40b0-986a-6fbff712a6f5" containerName="revoke-keys"
Sep 10 19:38:55.363127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.363108    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad8100a9-8ea1-40b0-986a-6fbff712a6f5" containerName="revoke-keys"
Sep 10 19:38:55.366469 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.366448    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.369486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.369434    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f5248185-dockercfg-d5z8v\""
Sep 10 19:38:55.369486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.369464    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f5248185\""
Sep 10 19:38:55.385817 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.385784    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"]
Sep 10 19:38:55.452985 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.452943    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42xrb\" (UniqueName: \"kubernetes.io/projected/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-kube-api-access-42xrb\") pod \"payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.453173 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.453088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.554083 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.553977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.554083 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.554058    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42xrb\" (UniqueName: \"kubernetes.io/projected/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-kube-api-access-42xrb\") pod \"payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.554700 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.554675    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.562344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.562308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42xrb\" (UniqueName: \"kubernetes.io/projected/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-kube-api-access-42xrb\") pod \"payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.677523 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.677480    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:55.955629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:55.955587    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"]
Sep 10 19:38:55.958698 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:38:55.958669    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896b44f7_e86b_41ba_9dff_a4c9a0bc9a58.slice/crio-1615e33588891107a4ff08a3c8e9de51e30a9135fdcd5f64390ceb99fd12a806 WatchSource:0}: Error finding container 1615e33588891107a4ff08a3c8e9de51e30a9135fdcd5f64390ceb99fd12a806: Status 404 returned error can't find the container with id 1615e33588891107a4ff08a3c8e9de51e30a9135fdcd5f64390ceb99fd12a806
Sep 10 19:38:56.376749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:56.376643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t" event={"ID":"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58","Type":"ContainerStarted","Data":"913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff"}
Sep 10 19:38:56.376749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:56.376707    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t" event={"ID":"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58","Type":"ContainerStarted","Data":"1615e33588891107a4ff08a3c8e9de51e30a9135fdcd5f64390ceb99fd12a806"}
Sep 10 19:38:56.376749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:56.376741    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:38:56.399345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:56.399290    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t" podStartSLOduration=1.399275658 podStartE2EDuration="1.399275658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:38:56.398268437 +0000 UTC m=+1129.338302271" watchObservedRunningTime="2026-09-10 19:38:56.399275658 +0000 UTC m=+1129.339309482"
Sep 10 19:38:57.487163 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:57.487122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:58.095210 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.095173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:58.141368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.141331    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"]
Sep 10 19:38:58.141579 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.141550    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z" podUID="f058e745-d4b9-4044-bec6-168545fdd5a3" containerName="payload-pre-processing" containerID="cri-o://28bd2bcc29484acdee5e0c7bde9c94e9751ad2e27918ab2e8b60b4fddbeb36fc" gracePeriod=30
Sep 10 19:38:58.197378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.197348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:58.288568 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.288531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:58.367303 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.367208    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"]
Sep 10 19:38:58.370188 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.370158    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a037c8-fxx6r"]
Sep 10 19:38:58.372021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.371994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:58.388959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.388921    2562 generic.go:356] "Generic (PLEG): container finished" podID="f058e745-d4b9-4044-bec6-168545fdd5a3" containerID="28bd2bcc29484acdee5e0c7bde9c94e9751ad2e27918ab2e8b60b4fddbeb36fc" exitCode=0
Sep 10 19:38:58.389075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.388992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z" event={"ID":"f058e745-d4b9-4044-bec6-168545fdd5a3","Type":"ContainerDied","Data":"28bd2bcc29484acdee5e0c7bde9c94e9751ad2e27918ab2e8b60b4fddbeb36fc"}
Sep 10 19:38:58.389075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.389055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z" event={"ID":"f058e745-d4b9-4044-bec6-168545fdd5a3","Type":"ContainerDied","Data":"f3955a7fc5de37845a798e2b80bd1915316710eed9548c6d26dff6e276ce0fe1"}
Sep 10 19:38:58.389164 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.389078    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3955a7fc5de37845a798e2b80bd1915316710eed9548c6d26dff6e276ce0fe1"
Sep 10 19:38:58.400410 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.400386    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:38:58.481906 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.481853    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f058e745-d4b9-4044-bec6-168545fdd5a3-plugins-config-volume\") pod \"f058e745-d4b9-4044-bec6-168545fdd5a3\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") "
Sep 10 19:38:58.482072 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.481948    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjdcq\" (UniqueName: \"kubernetes.io/projected/f058e745-d4b9-4044-bec6-168545fdd5a3-kube-api-access-bjdcq\") pod \"f058e745-d4b9-4044-bec6-168545fdd5a3\" (UID: \"f058e745-d4b9-4044-bec6-168545fdd5a3\") "
Sep 10 19:38:58.482249 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.482227    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f058e745-d4b9-4044-bec6-168545fdd5a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f058e745-d4b9-4044-bec6-168545fdd5a3" (UID: "f058e745-d4b9-4044-bec6-168545fdd5a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:38:58.484109 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.484085    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f058e745-d4b9-4044-bec6-168545fdd5a3-kube-api-access-bjdcq" (OuterVolumeSpecName: "kube-api-access-bjdcq") pod "f058e745-d4b9-4044-bec6-168545fdd5a3" (UID: "f058e745-d4b9-4044-bec6-168545fdd5a3"). InnerVolumeSpecName "kube-api-access-bjdcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:38:58.582609 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.582566    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjdcq\" (UniqueName: \"kubernetes.io/projected/f058e745-d4b9-4044-bec6-168545fdd5a3-kube-api-access-bjdcq\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:58.582609 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.582603    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f058e745-d4b9-4044-bec6-168545fdd5a3-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:38:58.746277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:58.746232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:38:59.394226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:59.394190    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"
Sep 10 19:38:59.421118 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:59.421084    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"]
Sep 10 19:38:59.425029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:59.424998    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a037c8-d46f8fd66-vrj7z"]
Sep 10 19:38:59.695356 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:59.695272    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad8100a9-8ea1-40b0-986a-6fbff712a6f5" path="/var/lib/kubelet/pods/ad8100a9-8ea1-40b0-986a-6fbff712a6f5/volumes"
Sep 10 19:38:59.695712 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:38:59.695647    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f058e745-d4b9-4044-bec6-168545fdd5a3" path="/var/lib/kubelet/pods/f058e745-d4b9-4044-bec6-168545fdd5a3/volumes"
Sep 10 19:39:04.664172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:04.664136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:04.761870 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:04.761832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:04.891243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:04.891203    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"]
Sep 10 19:39:04.891496 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:04.891465    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy" containerID="cri-o://9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998" gracePeriod=30
Sep 10 19:39:04.910003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:04.909962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:05.481836 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:05.481798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:05.780001 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:05.779878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:06.192763 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:06.192720    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:07.384263 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:07.384234    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:39:08.502223 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:08.502177    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:39:08.618728 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:08.618697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:08.705698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:08.705656    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:09.766151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:09.766115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:09.866489 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:09.866448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:09.965409 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:09.965369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:10.155369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.155337    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:39:10.274539 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.274488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:10.294366 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294331    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-podinfo\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294376    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istiod-ca-cert\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294434    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-socket\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294457    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-data\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294515    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-credential-socket\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294559    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-envoy\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294616    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-certs\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294645    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-token\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294686    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tv6b\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-kube-api-access-6tv6b\") pod \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\" (UID: \"0b863ade-c45e-45c0-8d00-d4cb4fff9546\") "
Sep 10 19:39:10.294814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:10.295038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294823    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:10.295038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294959    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:10.295038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.294997    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:10.295038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.295023    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.295246 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.295045    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-credential-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.295296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.295280    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:10.297149 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.297117    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:39:10.297149 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.297122    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:10.297321 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.297162    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:10.297321 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.297283    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-kube-api-access-6tv6b" (OuterVolumeSpecName: "kube-api-access-6tv6b") pod "0b863ade-c45e-45c0-8d00-d4cb4fff9546" (UID: "0b863ade-c45e-45c0-8d00-d4cb4fff9546"). InnerVolumeSpecName "kube-api-access-6tv6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:10.344986 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.344877    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:10.396189 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.396144    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-workload-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.396189 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.396182    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-token\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.396189 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.396191    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tv6b\" (UniqueName: \"kubernetes.io/projected/0b863ade-c45e-45c0-8d00-d4cb4fff9546-kube-api-access-6tv6b\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.396189 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.396199    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-podinfo\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.396471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.396209    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istiod-ca-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.396471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.396217    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-data\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.396471 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.396224    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b863ade-c45e-45c0-8d00-d4cb4fff9546-istio-envoy\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:10.409592 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.409556    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"]
Sep 10 19:39:10.410103 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.410087    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f058e745-d4b9-4044-bec6-168545fdd5a3" containerName="payload-pre-processing"
Sep 10 19:39:10.410159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.410106    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f058e745-d4b9-4044-bec6-168545fdd5a3" containerName="payload-pre-processing"
Sep 10 19:39:10.410159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.410123    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy"
Sep 10 19:39:10.410159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.410132    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy"
Sep 10 19:39:10.410255 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.410234    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy"
Sep 10 19:39:10.410255 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.410248    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f058e745-d4b9-4044-bec6-168545fdd5a3" containerName="payload-pre-processing"
Sep 10 19:39:10.413664 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.413640    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.416861 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.416835    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-07c5fe-dockercfg-clm7m\""
Sep 10 19:39:10.417017 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.416860    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-07c5fe\""
Sep 10 19:39:10.426327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.426294    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"]
Sep 10 19:39:10.453244 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.453207    2562 generic.go:356] "Generic (PLEG): container finished" podID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerID="9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998" exitCode=0
Sep 10 19:39:10.453399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.453270    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"
Sep 10 19:39:10.453399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.453285    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" event={"ID":"0b863ade-c45e-45c0-8d00-d4cb4fff9546","Type":"ContainerDied","Data":"9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998"}
Sep 10 19:39:10.453399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.453346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g" event={"ID":"0b863ade-c45e-45c0-8d00-d4cb4fff9546","Type":"ContainerDied","Data":"cbde327869959adc69c0e1f01957d37c5608ce9a259c4888a0c036272747e7e8"}
Sep 10 19:39:10.453399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.453364    2562 scope.go:117] "RemoveContainer" containerID="9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998"
Sep 10 19:39:10.455328 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.455306    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"]
Sep 10 19:39:10.455704 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.455689    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy"
Sep 10 19:39:10.455704 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.455705    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy"
Sep 10 19:39:10.455846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.455784    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" containerName="istio-proxy"
Sep 10 19:39:10.459864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.459842    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.463336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.463315    2562 scope.go:117] "RemoveContainer" containerID="5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c"
Sep 10 19:39:10.472760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.472724    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"]
Sep 10 19:39:10.473348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.473324    2562 scope.go:117] "RemoveContainer" containerID="9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998"
Sep 10 19:39:10.473757 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:39:10.473729    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998\": container with ID starting with 9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998 not found: ID does not exist" containerID="9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998"
Sep 10 19:39:10.473831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.473769    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998"} err="failed to get container status \"9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998\": rpc error: code = NotFound desc = could not find container \"9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998\": container with ID starting with 9dc65547c8f635edb8aa5082ac090594721c0a7975216cf6085381382e149998 not found: ID does not exist"
Sep 10 19:39:10.473831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.473794    2562 scope.go:117] "RemoveContainer" containerID="5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c"
Sep 10 19:39:10.474176 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:39:10.474157    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c\": container with ID starting with 5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c not found: ID does not exist" containerID="5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c"
Sep 10 19:39:10.474262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.474182    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c"} err="failed to get container status \"5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c\": rpc error: code = NotFound desc = could not find container \"5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c\": container with ID starting with 5fc32def6b3c8a863dace1b14da6c398d5fcc1c95e38ab36a961340d2f5e2f2c not found: ID does not exist"
Sep 10 19:39:10.500858 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.500818    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"]
Sep 10 19:39:10.505811 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.505778    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a037c8-openshift-default-79fb447fd4-frb2g"]
Sep 10 19:39:10.598908 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.598795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb8fc27-d99f-4a49-9f47-78999ac5becc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.598908 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.598841    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z92d\" (UniqueName: \"kubernetes.io/projected/5cb8fc27-d99f-4a49-9f47-78999ac5becc-kube-api-access-7z92d\") pod \"payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.599100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.598942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a534b13-f63b-4512-8283-a10d7c324f9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.599100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.598963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwdgb\" (UniqueName: \"kubernetes.io/projected/0a534b13-f63b-4512-8283-a10d7c324f9f-kube-api-access-vwdgb\") pod \"payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.700099 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.700065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a534b13-f63b-4512-8283-a10d7c324f9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.700248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.700106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwdgb\" (UniqueName: \"kubernetes.io/projected/0a534b13-f63b-4512-8283-a10d7c324f9f-kube-api-access-vwdgb\") pod \"payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.700248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.700218    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb8fc27-d99f-4a49-9f47-78999ac5becc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.700383 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.700249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z92d\" (UniqueName: \"kubernetes.io/projected/5cb8fc27-d99f-4a49-9f47-78999ac5becc-kube-api-access-7z92d\") pod \"payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.700867 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.700842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a534b13-f63b-4512-8283-a10d7c324f9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.700994 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.700959    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb8fc27-d99f-4a49-9f47-78999ac5becc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.711593 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.711560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z92d\" (UniqueName: \"kubernetes.io/projected/5cb8fc27-d99f-4a49-9f47-78999ac5becc-kube-api-access-7z92d\") pod \"payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.712076 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.712054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwdgb\" (UniqueName: \"kubernetes.io/projected/0a534b13-f63b-4512-8283-a10d7c324f9f-kube-api-access-vwdgb\") pod \"payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.723932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.723873    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:10.772524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.772411    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:10.998854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:10.998827    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"]
Sep 10 19:39:11.001357 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:39:11.001316    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb8fc27_d99f_4a49_9f47_78999ac5becc.slice/crio-34f4071c3c0f0cb7cb359f9e3d0e599d4a68ae6b6bf470b4771c0e70845d1f0f WatchSource:0}: Error finding container 34f4071c3c0f0cb7cb359f9e3d0e599d4a68ae6b6bf470b4771c0e70845d1f0f: Status 404 returned error can't find the container with id 34f4071c3c0f0cb7cb359f9e3d0e599d4a68ae6b6bf470b4771c0e70845d1f0f
Sep 10 19:39:11.062490 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.062455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"]
Sep 10 19:39:11.066332 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:39:11.066306    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a534b13_f63b_4512_8283_a10d7c324f9f.slice/crio-c1cc34f430e88e3a211a56f8e1f2aa703a5333be474f61c278e8567f35c69469 WatchSource:0}: Error finding container c1cc34f430e88e3a211a56f8e1f2aa703a5333be474f61c278e8567f35c69469: Status 404 returned error can't find the container with id c1cc34f430e88e3a211a56f8e1f2aa703a5333be474f61c278e8567f35c69469
Sep 10 19:39:11.460879 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.460833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq" event={"ID":"0a534b13-f63b-4512-8283-a10d7c324f9f","Type":"ContainerStarted","Data":"6c403310b303a94234faa7e3b46d8cd2f55fcfd270d29f86abbf7844a882ac1a"}
Sep 10 19:39:11.461117 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.461008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:11.461117 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.461048    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq" event={"ID":"0a534b13-f63b-4512-8283-a10d7c324f9f","Type":"ContainerStarted","Data":"c1cc34f430e88e3a211a56f8e1f2aa703a5333be474f61c278e8567f35c69469"}
Sep 10 19:39:11.463524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.463496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx" event={"ID":"5cb8fc27-d99f-4a49-9f47-78999ac5becc","Type":"ContainerStarted","Data":"906530d57edfc19478535c45fb4643b65cc48defe84956fc449490d0a95c918e"}
Sep 10 19:39:11.463682 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.463538    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx" event={"ID":"5cb8fc27-d99f-4a49-9f47-78999ac5becc","Type":"ContainerStarted","Data":"34f4071c3c0f0cb7cb359f9e3d0e599d4a68ae6b6bf470b4771c0e70845d1f0f"}
Sep 10 19:39:11.463682 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.463578    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:11.479467 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.479407    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq" podStartSLOduration=1.479386694 podStartE2EDuration="1.479386694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:11.479230002 +0000 UTC m=+1144.419263835" watchObservedRunningTime="2026-09-10 19:39:11.479386694 +0000 UTC m=+1144.419420533"
Sep 10 19:39:11.500223 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.500158    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx" podStartSLOduration=1.500139113 podStartE2EDuration="1.500139113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:11.498294517 +0000 UTC m=+1144.438328351" watchObservedRunningTime="2026-09-10 19:39:11.500139113 +0000 UTC m=+1144.440172946"
Sep 10 19:39:11.697303 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:11.697257    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b863ade-c45e-45c0-8d00-d4cb4fff9546" path="/var/lib/kubelet/pods/0b863ade-c45e-45c0-8d00-d4cb4fff9546/volumes"
Sep 10 19:39:15.503730 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:15.503668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:15.686665 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:15.686626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:18.373915 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:18.373195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:19.047103 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:19.047065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:19.140638 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:19.140603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:19.501475 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:19.501437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:21.107608 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:21.107565    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:39:22.471254 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:22.471216    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:39:22.471647 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:22.471443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:39:25.220016 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:25.219984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:25.320338 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:25.320300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:31.997013 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:31.996919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:33.578249 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:33.578194    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:33.905154 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:33.905119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:34.212610 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:34.212512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:34.780333 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:34.780296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:35.821550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:35.821511    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:36.506610 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.506573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:36.541258 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.541222    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"]
Sep 10 19:39:36.541504 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.541473    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t" podUID="896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" containerName="payload-pre-processing" containerID="cri-o://913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff" gracePeriod=30
Sep 10 19:39:36.607661 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.607623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:36.718758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.718723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:36.787493 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.787466    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:39:36.845238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.845194    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42xrb\" (UniqueName: \"kubernetes.io/projected/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-kube-api-access-42xrb\") pod \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") "
Sep 10 19:39:36.845238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.845244    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-plugins-config-volume\") pod \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\" (UID: \"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58\") "
Sep 10 19:39:36.845845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.845820    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" (UID: "896b44f7-e86b-41ba-9dff-a4c9a0bc9a58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:36.847618 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.847588    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-kube-api-access-42xrb" (OuterVolumeSpecName: "kube-api-access-42xrb") pod "896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" (UID: "896b44f7-e86b-41ba-9dff-a4c9a0bc9a58"). InnerVolumeSpecName "kube-api-access-42xrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:36.904642 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.904607    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:36.947059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.947018    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42xrb\" (UniqueName: \"kubernetes.io/projected/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-kube-api-access-42xrb\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:36.947059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:36.947050    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:39:37.000958 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.000924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:37.100592 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.100498    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:37.605053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.605017    2562 generic.go:356] "Generic (PLEG): container finished" podID="896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" containerID="913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff" exitCode=0
Sep 10 19:39:37.605231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.605061    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"
Sep 10 19:39:37.605231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.605104    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t" event={"ID":"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58","Type":"ContainerDied","Data":"913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff"}
Sep 10 19:39:37.605231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.605154    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t" event={"ID":"896b44f7-e86b-41ba-9dff-a4c9a0bc9a58","Type":"ContainerDied","Data":"1615e33588891107a4ff08a3c8e9de51e30a9135fdcd5f64390ceb99fd12a806"}
Sep 10 19:39:37.605231 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.605179    2562 scope.go:117] "RemoveContainer" containerID="913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff"
Sep 10 19:39:37.614692 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.614670    2562 scope.go:117] "RemoveContainer" containerID="913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff"
Sep 10 19:39:37.615015 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:39:37.614992    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff\": container with ID starting with 913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff not found: ID does not exist" containerID="913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff"
Sep 10 19:39:37.615101 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.615029    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff"} err="failed to get container status \"913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff\": rpc error: code = NotFound desc = could not find container \"913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff\": container with ID starting with 913dab969c946176b40f0d3c28a3b7c1782e1e072f9f17ef5679992b482fedff not found: ID does not exist"
Sep 10 19:39:37.627428 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.627395    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"]
Sep 10 19:39:37.630849 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.630821    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f5248185-84d976dd6-4s94t"]
Sep 10 19:39:37.697075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:37.697041    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" path="/var/lib/kubelet/pods/896b44f7-e86b-41ba-9dff-a4c9a0bc9a58/volumes"
Sep 10 19:39:38.078191 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.078139    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:38.180299 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.180261    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:38.299374 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.299339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:38.559508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.559470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:38.657687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.657640    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:38.700199 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.700159    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"]
Sep 10 19:39:38.700567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.700554    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" containerName="payload-pre-processing"
Sep 10 19:39:38.700609 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.700568    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" containerName="payload-pre-processing"
Sep 10 19:39:38.700648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.700643    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="896b44f7-e86b-41ba-9dff-a4c9a0bc9a58" containerName="payload-pre-processing"
Sep 10 19:39:38.705240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.705218    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:38.710002 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.709975    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-2a1fa9\""
Sep 10 19:39:38.710237 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.710222    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-2a1fa9-dockercfg-lbv8t\""
Sep 10 19:39:38.718054 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.718024    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"]
Sep 10 19:39:38.765870 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.765832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzm5s\" (UniqueName: \"kubernetes.io/projected/d79696b1-fe74-4685-a4aa-40ab12471d90-kube-api-access-kzm5s\") pod \"payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:38.766032 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.765925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79696b1-fe74-4685-a4aa-40ab12471d90-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:38.867341 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.867247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79696b1-fe74-4685-a4aa-40ab12471d90-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:38.867341 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.867336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzm5s\" (UniqueName: \"kubernetes.io/projected/d79696b1-fe74-4685-a4aa-40ab12471d90-kube-api-access-kzm5s\") pod \"payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:38.867983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.867962    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79696b1-fe74-4685-a4aa-40ab12471d90-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:38.878419 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:38.878324    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzm5s\" (UniqueName: \"kubernetes.io/projected/d79696b1-fe74-4685-a4aa-40ab12471d90-kube-api-access-kzm5s\") pod \"payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:39.016357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:39.016317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:39.301964 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:39.301719    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"]
Sep 10 19:39:39.304465 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:39:39.304440    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79696b1_fe74_4685_a4aa_40ab12471d90.slice/crio-7898417fa0fbb0d52072b63e7af17883ab7c3bad50e329e0bff0c433c8a3474e WatchSource:0}: Error finding container 7898417fa0fbb0d52072b63e7af17883ab7c3bad50e329e0bff0c433c8a3474e: Status 404 returned error can't find the container with id 7898417fa0fbb0d52072b63e7af17883ab7c3bad50e329e0bff0c433c8a3474e
Sep 10 19:39:39.617820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:39.617725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq" event={"ID":"d79696b1-fe74-4685-a4aa-40ab12471d90","Type":"ContainerStarted","Data":"7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060"}
Sep 10 19:39:39.617820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:39.617787    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:39.617820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:39.617812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq" event={"ID":"d79696b1-fe74-4685-a4aa-40ab12471d90","Type":"ContainerStarted","Data":"7898417fa0fbb0d52072b63e7af17883ab7c3bad50e329e0bff0c433c8a3474e"}
Sep 10 19:39:39.638277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:39.638215    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq" podStartSLOduration=1.6381959 podStartE2EDuration="1.6381959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:39.636496953 +0000 UTC m=+1172.576530788" watchObservedRunningTime="2026-09-10 19:39:39.6381959 +0000 UTC m=+1172.578229733"
Sep 10 19:39:42.812702 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:42.812664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:39:42.959406 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:42.959369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:43.073483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:43.073364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:43.255739 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:43.255701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:43.595534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:43.595496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:43.694680 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:43.694643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:43.805243 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:43.805186    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:49.907598 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:49.907549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:50.018024 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:50.017985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:50.109656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:50.109608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:50.261007 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:50.260910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:50.625737 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:50.625652    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:39:54.811420 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:54.811379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:56.072685 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:56.072645    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:56.135266 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:56.135220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:56.991194 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:56.991160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:57.306062 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:57.305956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:39:57.639339 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:39:57.639298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:01.015152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:01.015115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:01.113771 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:01.113727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:01.869223 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:01.869179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:02.194105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:02.194075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:02.277965 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:02.277926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:02.449987 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:02.449869    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:02.929220 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:02.929178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:03.032558 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.032522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:03.129735 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.129697    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"]
Sep 10 19:40:03.134647 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.134627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.137556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.137532    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-ecadb8\""
Sep 10 19:40:03.137854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.137837    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-ecadb8-dockercfg-h5n29\""
Sep 10 19:40:03.143823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.143797    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"]
Sep 10 19:40:03.295707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.295606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7afec082-517e-431d-8f16-41bb6b086b88-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.295707 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.295675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whhm4\" (UniqueName: \"kubernetes.io/projected/7afec082-517e-431d-8f16-41bb6b086b88-kube-api-access-whhm4\") pod \"payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.397086 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.397041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7afec082-517e-431d-8f16-41bb6b086b88-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.397273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.397112    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whhm4\" (UniqueName: \"kubernetes.io/projected/7afec082-517e-431d-8f16-41bb6b086b88-kube-api-access-whhm4\") pod \"payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.397759 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.397739    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7afec082-517e-431d-8f16-41bb6b086b88-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.405097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.405067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whhm4\" (UniqueName: \"kubernetes.io/projected/7afec082-517e-431d-8f16-41bb6b086b88-kube-api-access-whhm4\") pod \"payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.445940 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.445879    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:03.723863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.723832    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"]
Sep 10 19:40:03.726325 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:40:03.726297    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7afec082_517e_431d_8f16_41bb6b086b88.slice/crio-ff74ca10c77edb32a5dbed2f121606898ac20e9c4dbef061ab8a37ed02914fad WatchSource:0}: Error finding container ff74ca10c77edb32a5dbed2f121606898ac20e9c4dbef061ab8a37ed02914fad: Status 404 returned error can't find the container with id ff74ca10c77edb32a5dbed2f121606898ac20e9c4dbef061ab8a37ed02914fad
Sep 10 19:40:03.748351 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:03.748320    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx" event={"ID":"7afec082-517e-431d-8f16-41bb6b086b88","Type":"ContainerStarted","Data":"ff74ca10c77edb32a5dbed2f121606898ac20e9c4dbef061ab8a37ed02914fad"}
Sep 10 19:40:04.755991 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:04.755950    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx" event={"ID":"7afec082-517e-431d-8f16-41bb6b086b88","Type":"ContainerStarted","Data":"b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17"}
Sep 10 19:40:04.756411 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:04.756048    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:04.773939 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:04.773860    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx" podStartSLOduration=1.773842082 podStartE2EDuration="1.773842082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:04.771578601 +0000 UTC m=+1197.711612457" watchObservedRunningTime="2026-09-10 19:40:04.773842082 +0000 UTC m=+1197.713875913"
Sep 10 19:40:06.901868 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:06.901835    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:40:07.619662 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:07.619635    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:40:07.624394 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:07.624371    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:40:07.776110 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:07.776079    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:40:07.781482 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:07.781452    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:40:15.763484 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:15.763452    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:40:17.165978 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.165939    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"]
Sep 10 19:40:17.169676 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.169653    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.174323 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.174288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"openshift-service-ca.crt\""
Sep 10 19:40:17.174323 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.174296    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"kube-root-ca.crt\""
Sep 10 19:40:17.174549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.174296    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"auth-test-model-07c5fe-kserve-self-signed-certs\""
Sep 10 19:40:17.174549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.174336    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"default-dockercfg-6ktvk\""
Sep 10 19:40:17.180100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.180073    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"]
Sep 10 19:40:17.198350 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.198308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:17.212408 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.212363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-home\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.212408 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.212411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rht2l\" (UniqueName: \"kubernetes.io/projected/58433946-c553-4b13-968e-6b769fa4e936-kube-api-access-rht2l\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.212631 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.212490    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-model-cache\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.212631 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.212548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58433946-c553-4b13-968e-6b769fa4e936-tls-certs\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.212710 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.212645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-tmp-dir\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.212710 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.212690    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-dshm\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313234 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-dshm\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313234 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-home\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313503 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313252    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rht2l\" (UniqueName: \"kubernetes.io/projected/58433946-c553-4b13-968e-6b769fa4e936-kube-api-access-rht2l\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313503 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-model-cache\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313503 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58433946-c553-4b13-968e-6b769fa4e936-tls-certs\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313503 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313414    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-tmp-dir\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-home\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-model-cache\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.313922 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.313793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-tmp-dir\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.315621 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.315590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-dshm\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.315874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.315856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58433946-c553-4b13-968e-6b769fa4e936-tls-certs\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.322344 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.322314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rht2l\" (UniqueName: \"kubernetes.io/projected/58433946-c553-4b13-968e-6b769fa4e936-kube-api-access-rht2l\") pod \"auth-test-model-07c5fe-kserve-74458dbfdc-t9prm\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.480608 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.480489    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:17.766228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.766145    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"]
Sep 10 19:40:17.768634 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:40:17.768605    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58433946_c553_4b13_968e_6b769fa4e936.slice/crio-f165d6583a733e4b9d4485564df81c492f395842552dcf43cc8d075d4e3b71af WatchSource:0}: Error finding container f165d6583a733e4b9d4485564df81c492f395842552dcf43cc8d075d4e3b71af: Status 404 returned error can't find the container with id f165d6583a733e4b9d4485564df81c492f395842552dcf43cc8d075d4e3b71af
Sep 10 19:40:17.829738 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.829706    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm" event={"ID":"58433946-c553-4b13-968e-6b769fa4e936","Type":"ContainerStarted","Data":"b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65"}
Sep 10 19:40:17.829738 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.829749    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm" event={"ID":"58433946-c553-4b13-968e-6b769fa4e936","Type":"ContainerStarted","Data":"f165d6583a733e4b9d4485564df81c492f395842552dcf43cc8d075d4e3b71af"}
Sep 10 19:40:17.849074 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:17.849008    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm" podStartSLOduration=0.848991096 podStartE2EDuration="848.991096ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:17.846650668 +0000 UTC m=+1210.786684504" watchObservedRunningTime="2026-09-10 19:40:17.848991096 +0000 UTC m=+1210.789024922"
Sep 10 19:40:22.513651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:22.513606    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:40:25.814279 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:25.814237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:26.154138 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:26.154103    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:26.471282 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:26.471182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:26.557455 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:26.557419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:27.481291 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:27.481251    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:27.481733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:27.481307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:27.494297 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:27.494265    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:27.894171 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:27.894133    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:40:29.074316 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:29.074273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:29.170337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:29.170284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:30.347561 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:30.347527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:30.444529 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:30.444484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:30.838100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:30.838064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:30.930395 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:30.930362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:31.132422 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:31.132321    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:37.003075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:37.003033    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:40:54.064465 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.064422    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:40:54.827037 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.827003    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7733-b57863"]
Sep 10 19:40:54.836051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.836022    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:40:54.838811 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.838781    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zp7h9\""
Sep 10 19:40:54.840235 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.840151    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-b57863"]
Sep 10 19:40:54.859989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.859954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9xwn\" (UniqueName: \"kubernetes.io/projected/71a0a4e0-3a1e-411f-bff3-38ab1579d966-kube-api-access-s9xwn\") pod \"test-curl-7733-b57863\" (UID: \"71a0a4e0-3a1e-411f-bff3-38ab1579d966\") " pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:40:54.960588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.960543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9xwn\" (UniqueName: \"kubernetes.io/projected/71a0a4e0-3a1e-411f-bff3-38ab1579d966-kube-api-access-s9xwn\") pod \"test-curl-7733-b57863\" (UID: \"71a0a4e0-3a1e-411f-bff3-38ab1579d966\") " pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:40:54.969069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:54.969030    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9xwn\" (UniqueName: \"kubernetes.io/projected/71a0a4e0-3a1e-411f-bff3-38ab1579d966-kube-api-access-s9xwn\") pod \"test-curl-7733-b57863\" (UID: \"71a0a4e0-3a1e-411f-bff3-38ab1579d966\") " pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:40:55.148151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:55.148122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:40:55.419438 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:55.419413    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-b57863"]
Sep 10 19:40:55.422100 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:40:55.422071    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71a0a4e0_3a1e_411f_bff3_38ab1579d966.slice/crio-599933541c28e82eb29c5db551b8d73f1c949aee759a5e2a93cd023cd9914f60 WatchSource:0}: Error finding container 599933541c28e82eb29c5db551b8d73f1c949aee759a5e2a93cd023cd9914f60: Status 404 returned error can't find the container with id 599933541c28e82eb29c5db551b8d73f1c949aee759a5e2a93cd023cd9914f60
Sep 10 19:40:56.035793 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.035744    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-b57863" event={"ID":"71a0a4e0-3a1e-411f-bff3-38ab1579d966","Type":"ContainerStarted","Data":"678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da"}
Sep 10 19:40:56.035793 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.035801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-b57863" event={"ID":"71a0a4e0-3a1e-411f-bff3-38ab1579d966","Type":"ContainerStarted","Data":"599933541c28e82eb29c5db551b8d73f1c949aee759a5e2a93cd023cd9914f60"}
Sep 10 19:40:56.051634 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.051572    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7733-b57863" podStartSLOduration=2.051553359 podStartE2EDuration="2.051553359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:56.050290019 +0000 UTC m=+1248.990323855" watchObservedRunningTime="2026-09-10 19:40:56.051553359 +0000 UTC m=+1248.991587195"
Sep 10 19:40:56.670333 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.670296    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7733-b57863"]
Sep 10 19:40:56.676378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.676347    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7733-b57863"]
Sep 10 19:40:56.789177 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.789095    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7733-17e77c"]
Sep 10 19:40:56.789494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.789481    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a0a4e0-3a1e-411f-bff3-38ab1579d966" containerName="test-curl-7733-b57863"
Sep 10 19:40:56.789537 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.789495    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a0a4e0-3a1e-411f-bff3-38ab1579d966" containerName="test-curl-7733-b57863"
Sep 10 19:40:56.789586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.789576    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a0a4e0-3a1e-411f-bff3-38ab1579d966" containerName="test-curl-7733-b57863"
Sep 10 19:40:56.792942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.792921    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:40:56.797369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.797333    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-17e77c"]
Sep 10 19:40:56.878680 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.878639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr4zs\" (UniqueName: \"kubernetes.io/projected/0f28445a-6bae-4a67-bf6e-7e727f69ce34-kube-api-access-kr4zs\") pod \"test-curl-7733-17e77c\" (UID: \"0f28445a-6bae-4a67-bf6e-7e727f69ce34\") " pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:40:56.979351 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.979303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr4zs\" (UniqueName: \"kubernetes.io/projected/0f28445a-6bae-4a67-bf6e-7e727f69ce34-kube-api-access-kr4zs\") pod \"test-curl-7733-17e77c\" (UID: \"0f28445a-6bae-4a67-bf6e-7e727f69ce34\") " pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:40:56.988029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:56.988001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr4zs\" (UniqueName: \"kubernetes.io/projected/0f28445a-6bae-4a67-bf6e-7e727f69ce34-kube-api-access-kr4zs\") pod \"test-curl-7733-17e77c\" (UID: \"0f28445a-6bae-4a67-bf6e-7e727f69ce34\") " pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:40:57.041671 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:40:57.041584    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7733-b57863" container="test-curl-7733-b57863"
Sep 10 19:40:57.103677 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:57.103636    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:40:57.374140 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:57.374110    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-17e77c"]
Sep 10 19:40:57.376855 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:40:57.376826    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f28445a_6bae_4a67_bf6e_7e727f69ce34.slice/crio-201e8757c54d064e3a9f6b9036a47f84638dc24bb6e3f24c9b3113445d73d624 WatchSource:0}: Error finding container 201e8757c54d064e3a9f6b9036a47f84638dc24bb6e3f24c9b3113445d73d624: Status 404 returned error can't find the container with id 201e8757c54d064e3a9f6b9036a47f84638dc24bb6e3f24c9b3113445d73d624
Sep 10 19:40:58.047921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.047870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-17e77c" event={"ID":"0f28445a-6bae-4a67-bf6e-7e727f69ce34","Type":"ContainerStarted","Data":"d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605"}
Sep 10 19:40:58.048394 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.047932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-17e77c" event={"ID":"0f28445a-6bae-4a67-bf6e-7e727f69ce34","Type":"ContainerStarted","Data":"201e8757c54d064e3a9f6b9036a47f84638dc24bb6e3f24c9b3113445d73d624"}
Sep 10 19:40:58.048394 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.047960    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7733-b57863" podUID="71a0a4e0-3a1e-411f-bff3-38ab1579d966" containerName="test-curl-7733-b57863" containerID="cri-o://678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da" gracePeriod=2
Sep 10 19:40:58.074493 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.074432    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7733-17e77c" podStartSLOduration=2.074415205 podStartE2EDuration="2.074415205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:58.072767698 +0000 UTC m=+1251.012801534" watchObservedRunningTime="2026-09-10 19:40:58.074415205 +0000 UTC m=+1251.014449046"
Sep 10 19:40:58.695566 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.695531    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7733-17e77c"]
Sep 10 19:40:58.699028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.698994    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7733-17e77c"]
Sep 10 19:40:58.814779 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.814692    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7733-256a28"]
Sep 10 19:40:58.815254 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.815231    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" containerName="test-curl-7733-17e77c"
Sep 10 19:40:58.815381 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.815258    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" containerName="test-curl-7733-17e77c"
Sep 10 19:40:58.815381 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.815348    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" containerName="test-curl-7733-17e77c"
Sep 10 19:40:58.819015 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.818995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:40:58.828106 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.828038    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-256a28"]
Sep 10 19:40:58.898131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.898065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sts42\" (UniqueName: \"kubernetes.io/projected/5ba2ecac-51e9-491a-a01b-fc8e74ab6e42-kube-api-access-sts42\") pod \"test-curl-7733-256a28\" (UID: \"5ba2ecac-51e9-491a-a01b-fc8e74ab6e42\") " pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:40:58.999663 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:58.999610    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sts42\" (UniqueName: \"kubernetes.io/projected/5ba2ecac-51e9-491a-a01b-fc8e74ab6e42-kube-api-access-sts42\") pod \"test-curl-7733-256a28\" (UID: \"5ba2ecac-51e9-491a-a01b-fc8e74ab6e42\") " pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:40:59.008864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:59.008839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sts42\" (UniqueName: \"kubernetes.io/projected/5ba2ecac-51e9-491a-a01b-fc8e74ab6e42-kube-api-access-sts42\") pod \"test-curl-7733-256a28\" (UID: \"5ba2ecac-51e9-491a-a01b-fc8e74ab6e42\") " pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:40:59.053179 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:40:59.053142    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7733-17e77c" container="test-curl-7733-17e77c"
Sep 10 19:40:59.139514 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:59.139482    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:40:59.415701 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:40:59.415675    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-256a28"]
Sep 10 19:40:59.417677 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:40:59.417646    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ba2ecac_51e9_491a_a01b_fc8e74ab6e42.slice/crio-93e567c5f5bb55feb31436aee42467850a71bfcd1aaeb84badd9b618cde05ffa WatchSource:0}: Error finding container 93e567c5f5bb55feb31436aee42467850a71bfcd1aaeb84badd9b618cde05ffa: Status 404 returned error can't find the container with id 93e567c5f5bb55feb31436aee42467850a71bfcd1aaeb84badd9b618cde05ffa
Sep 10 19:41:00.060751 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.060704    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-256a28" event={"ID":"5ba2ecac-51e9-491a-a01b-fc8e74ab6e42","Type":"ContainerStarted","Data":"6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899"}
Sep 10 19:41:00.061176 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.060765    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-256a28" event={"ID":"5ba2ecac-51e9-491a-a01b-fc8e74ab6e42","Type":"ContainerStarted","Data":"93e567c5f5bb55feb31436aee42467850a71bfcd1aaeb84badd9b618cde05ffa"}
Sep 10 19:41:00.061176 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.060761    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7733-17e77c" podUID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" containerName="test-curl-7733-17e77c" containerID="cri-o://d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605" gracePeriod=2
Sep 10 19:41:00.081497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.081441    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7733-256a28" podStartSLOduration=2.081424042 podStartE2EDuration="2.081424042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:00.079984662 +0000 UTC m=+1253.020018496" watchObservedRunningTime="2026-09-10 19:41:00.081424042 +0000 UTC m=+1253.021457915"
Sep 10 19:41:00.228031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.228003    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:41:00.312430 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.312333    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9xwn\" (UniqueName: \"kubernetes.io/projected/71a0a4e0-3a1e-411f-bff3-38ab1579d966-kube-api-access-s9xwn\") pod \"71a0a4e0-3a1e-411f-bff3-38ab1579d966\" (UID: \"71a0a4e0-3a1e-411f-bff3-38ab1579d966\") "
Sep 10 19:41:00.314387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.314348    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a0a4e0-3a1e-411f-bff3-38ab1579d966-kube-api-access-s9xwn" (OuterVolumeSpecName: "kube-api-access-s9xwn") pod "71a0a4e0-3a1e-411f-bff3-38ab1579d966" (UID: "71a0a4e0-3a1e-411f-bff3-38ab1579d966"). InnerVolumeSpecName "kube-api-access-s9xwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:00.413355 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.413319    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9xwn\" (UniqueName: \"kubernetes.io/projected/71a0a4e0-3a1e-411f-bff3-38ab1579d966-kube-api-access-s9xwn\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:41:00.685240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.685201    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7733-256a28"]
Sep 10 19:41:00.689902 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.689858    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7733-256a28"]
Sep 10 19:41:00.814241 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.814204    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7733-fad49d"]
Sep 10 19:41:00.814622 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.814609    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" containerName="test-curl-7733-256a28"
Sep 10 19:41:00.814669 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.814624    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" containerName="test-curl-7733-256a28"
Sep 10 19:41:00.814704 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.814687    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" containerName="test-curl-7733-256a28"
Sep 10 19:41:00.817850 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.817832    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:00.822272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.822240    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-fad49d"]
Sep 10 19:41:00.917824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:00.917778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fphn2\" (UniqueName: \"kubernetes.io/projected/da0664e4-56c6-48ca-8eb4-dcce53b06606-kube-api-access-fphn2\") pod \"test-curl-7733-fad49d\" (UID: \"da0664e4-56c6-48ca-8eb4-dcce53b06606\") " pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:01.019385 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.019287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fphn2\" (UniqueName: \"kubernetes.io/projected/da0664e4-56c6-48ca-8eb4-dcce53b06606-kube-api-access-fphn2\") pod \"test-curl-7733-fad49d\" (UID: \"da0664e4-56c6-48ca-8eb4-dcce53b06606\") " pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:01.028206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.028168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fphn2\" (UniqueName: \"kubernetes.io/projected/da0664e4-56c6-48ca-8eb4-dcce53b06606-kube-api-access-fphn2\") pod \"test-curl-7733-fad49d\" (UID: \"da0664e4-56c6-48ca-8eb4-dcce53b06606\") " pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:01.067694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.067650    2562 generic.go:356] "Generic (PLEG): container finished" podID="71a0a4e0-3a1e-411f-bff3-38ab1579d966" containerID="678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da" exitCode=137
Sep 10 19:41:01.067694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.067694    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:41:01.068175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.067747    2562 scope.go:117] "RemoveContainer" containerID="678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da"
Sep 10 19:41:01.068175 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:01.068016    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7733-256a28" container="test-curl-7733-256a28"
Sep 10 19:41:01.079928 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:01.079753    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7733-b57863\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="71a0a4e0-3a1e-411f-bff3-38ab1579d966" pod="opendatahub/test-curl-7733-b57863"
Sep 10 19:41:01.081054 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.081030    2562 scope.go:117] "RemoveContainer" containerID="678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da"
Sep 10 19:41:01.081415 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:01.081393    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da\": container with ID starting with 678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da not found: ID does not exist" containerID="678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da"
Sep 10 19:41:01.081470 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.081428    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da"} err="failed to get container status \"678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da\": rpc error: code = NotFound desc = could not find container \"678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da\": container with ID starting with 678c46a918b6f561c144e2352b09498cd32bd1a1d935d35b938e11cdc86703da not found: ID does not exist"
Sep 10 19:41:01.128566 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.128529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:01.401219 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.401160    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7733-fad49d"]
Sep 10 19:41:01.403697 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:41:01.403665    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda0664e4_56c6_48ca_8eb4_dcce53b06606.slice/crio-de13fa737fcb03acff5842973555b18c40428b462522c0c22e1e9c9a15e92fef WatchSource:0}: Error finding container de13fa737fcb03acff5842973555b18c40428b462522c0c22e1e9c9a15e92fef: Status 404 returned error can't find the container with id de13fa737fcb03acff5842973555b18c40428b462522c0c22e1e9c9a15e92fef
Sep 10 19:41:01.696797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:01.696705    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71a0a4e0-3a1e-411f-bff3-38ab1579d966" path="/var/lib/kubelet/pods/71a0a4e0-3a1e-411f-bff3-38ab1579d966/volumes"
Sep 10 19:41:02.075197 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.075150    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-fad49d" event={"ID":"da0664e4-56c6-48ca-8eb4-dcce53b06606","Type":"ContainerStarted","Data":"5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc"}
Sep 10 19:41:02.075559 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.075220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7733-fad49d" event={"ID":"da0664e4-56c6-48ca-8eb4-dcce53b06606","Type":"ContainerStarted","Data":"de13fa737fcb03acff5842973555b18c40428b462522c0c22e1e9c9a15e92fef"}
Sep 10 19:41:02.076483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.076453    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7733-256a28" podUID="5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" containerName="test-curl-7733-256a28" containerID="cri-o://6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899" gracePeriod=2
Sep 10 19:41:02.090387 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.090284    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7733-fad49d" podStartSLOduration=2.090267568 podStartE2EDuration="2.090267568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:02.089352499 +0000 UTC m=+1255.029386331" watchObservedRunningTime="2026-09-10 19:41:02.090267568 +0000 UTC m=+1255.030301440"
Sep 10 19:41:02.258303 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.258268    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:41:02.330681 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.330590    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr4zs\" (UniqueName: \"kubernetes.io/projected/0f28445a-6bae-4a67-bf6e-7e727f69ce34-kube-api-access-kr4zs\") pod \"0f28445a-6bae-4a67-bf6e-7e727f69ce34\" (UID: \"0f28445a-6bae-4a67-bf6e-7e727f69ce34\") "
Sep 10 19:41:02.332661 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.332633    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f28445a-6bae-4a67-bf6e-7e727f69ce34-kube-api-access-kr4zs" (OuterVolumeSpecName: "kube-api-access-kr4zs") pod "0f28445a-6bae-4a67-bf6e-7e727f69ce34" (UID: "0f28445a-6bae-4a67-bf6e-7e727f69ce34"). InnerVolumeSpecName "kube-api-access-kr4zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:02.432206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.432158    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr4zs\" (UniqueName: \"kubernetes.io/projected/0f28445a-6bae-4a67-bf6e-7e727f69ce34-kube-api-access-kr4zs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:41:02.573758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.573718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:02.715798 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.715763    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7733-fad49d"]
Sep 10 19:41:02.719481 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:02.719445    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7733-fad49d"]
Sep 10 19:41:03.082687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.082594    2562 generic.go:356] "Generic (PLEG): container finished" podID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" containerID="d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605" exitCode=137
Sep 10 19:41:03.082687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.082640    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:41:03.083200 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.082694    2562 scope.go:117] "RemoveContainer" containerID="d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605"
Sep 10 19:41:03.083200 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:03.083008    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7733-fad49d" container="test-curl-7733-fad49d"
Sep 10 19:41:03.092142 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.092124    2562 scope.go:117] "RemoveContainer" containerID="d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605"
Sep 10 19:41:03.092434 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:03.092414    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605\": container with ID starting with d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605 not found: ID does not exist" containerID="d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605"
Sep 10 19:41:03.092499 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.092443    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605"} err="failed to get container status \"d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605\": rpc error: code = NotFound desc = could not find container \"d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605\": container with ID starting with d5b3df67c4c228e0245573f60e67d1ce2153b7edfc3433bdb53aaebd5c322605 not found: ID does not exist"
Sep 10 19:41:03.093851 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:03.093825    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7733-17e77c\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:41:03.301683 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.301652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:03.573937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.573874    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"]
Sep 10 19:41:03.574531 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.574510    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da0664e4-56c6-48ca-8eb4-dcce53b06606" containerName="test-curl-7733-fad49d"
Sep 10 19:41:03.574651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.574533    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="da0664e4-56c6-48ca-8eb4-dcce53b06606" containerName="test-curl-7733-fad49d"
Sep 10 19:41:03.574651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.574623    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="da0664e4-56c6-48ca-8eb4-dcce53b06606" containerName="test-curl-7733-fad49d"
Sep 10 19:41:03.579601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.579575    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.583849 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.583825    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b97a9c60-openshift-default-dockercfg-rrtm7\""
Sep 10 19:41:03.584093 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:03.584063    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7733-17e77c\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" pod="opendatahub/test-curl-7733-17e77c"
Sep 10 19:41:03.584174 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.584076    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 19:41:03.598414 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.598379    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"]
Sep 10 19:41:03.606454 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.606416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:03.643820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.643784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-socket\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.643820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.643827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-envoy\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.644121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.643919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istiod-ca-cert\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.644121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.643951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-credential-socket\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.644121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.643999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-podinfo\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.644121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.644027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-data\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.644121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.644052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-token\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.644351 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.644157    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-certs\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.644351 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.644201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtff2\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-kube-api-access-rtff2\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.695759 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.695711    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f28445a-6bae-4a67-bf6e-7e727f69ce34" path="/var/lib/kubelet/pods/0f28445a-6bae-4a67-bf6e-7e727f69ce34/volumes"
Sep 10 19:41:03.744691 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.744660    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-socket\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.744691 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.744695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-envoy\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.744921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.744724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istiod-ca-cert\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.744921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.744857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-credential-socket\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745055 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.744951    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-podinfo\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745055 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.744988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-data\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745055 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-token\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-socket\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-certs\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745196    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-credential-socket\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtff2\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-kube-api-access-rtff2\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745302    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-data\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-certs\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.745562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.745466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istiod-ca-cert\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.747348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.747320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-envoy\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.747552 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.747534    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-podinfo\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.753413 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.753389    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtff2\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-kube-api-access-rtff2\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.753513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.753435    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-token\") pod \"e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") " pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:03.864094 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.863987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:03.890788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:03.890748    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:04.090398 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.090361    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7733-fad49d" podUID="da0664e4-56c6-48ca-8eb4-dcce53b06606" containerName="test-curl-7733-fad49d" containerID="cri-o://5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc" gracePeriod=2
Sep 10 19:41:04.173661 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.173627    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"]
Sep 10 19:41:04.176361 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:41:04.176324    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949ec5a0_f9ac_47a4_ab61_3f857c94ca41.slice/crio-bde9691307c914edeb0b353c8e5b819ca4dd02746cf3fbc14942961a8d12da1e WatchSource:0}: Error finding container bde9691307c914edeb0b353c8e5b819ca4dd02746cf3fbc14942961a8d12da1e: Status 404 returned error can't find the container with id bde9691307c914edeb0b353c8e5b819ca4dd02746cf3fbc14942961a8d12da1e
Sep 10 19:41:04.178865 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.178830    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:41:04.178985 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.178924    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:41:04.178985 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.178959    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:41:04.254256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.254233    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:41:04.350052 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.350022    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sts42\" (UniqueName: \"kubernetes.io/projected/5ba2ecac-51e9-491a-a01b-fc8e74ab6e42-kube-api-access-sts42\") pod \"5ba2ecac-51e9-491a-a01b-fc8e74ab6e42\" (UID: \"5ba2ecac-51e9-491a-a01b-fc8e74ab6e42\") "
Sep 10 19:41:04.352331 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.352296    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba2ecac-51e9-491a-a01b-fc8e74ab6e42-kube-api-access-sts42" (OuterVolumeSpecName: "kube-api-access-sts42") pod "5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" (UID: "5ba2ecac-51e9-491a-a01b-fc8e74ab6e42"). InnerVolumeSpecName "kube-api-access-sts42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:04.451551 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:04.451518    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sts42\" (UniqueName: \"kubernetes.io/projected/5ba2ecac-51e9-491a-a01b-fc8e74ab6e42-kube-api-access-sts42\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:41:05.099546 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.099438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" event={"ID":"949ec5a0-f9ac-47a4-ab61-3f857c94ca41","Type":"ContainerStarted","Data":"1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12"}
Sep 10 19:41:05.099546 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.099512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" event={"ID":"949ec5a0-f9ac-47a4-ab61-3f857c94ca41","Type":"ContainerStarted","Data":"bde9691307c914edeb0b353c8e5b819ca4dd02746cf3fbc14942961a8d12da1e"}
Sep 10 19:41:05.101224 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.101195    2562 generic.go:356] "Generic (PLEG): container finished" podID="5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" containerID="6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899" exitCode=137
Sep 10 19:41:05.101377 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.101231    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:41:05.101377 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.101266    2562 scope.go:117] "RemoveContainer" containerID="6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899"
Sep 10 19:41:05.111050 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.111031    2562 scope.go:117] "RemoveContainer" containerID="6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899"
Sep 10 19:41:05.111389 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:05.111367    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899\": container with ID starting with 6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899 not found: ID does not exist" containerID="6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899"
Sep 10 19:41:05.111446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.111398    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899"} err="failed to get container status \"6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899\": rpc error: code = NotFound desc = could not find container \"6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899\": container with ID starting with 6f699ee46f571702b18b7b1ae7e87cf450cd26c52aa91ab2f55fe41f8f304899 not found: ID does not exist"
Sep 10 19:41:05.121490 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.121440    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podStartSLOduration=2.121427572 podStartE2EDuration="2.121427572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:05.119678523 +0000 UTC m=+1258.059712356" watchObservedRunningTime="2026-09-10 19:41:05.121427572 +0000 UTC m=+1258.061461404"
Sep 10 19:41:05.121796 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:05.121768    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7733-256a28\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" pod="opendatahub/test-curl-7733-256a28"
Sep 10 19:41:05.697369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.697326    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ba2ecac-51e9-491a-a01b-fc8e74ab6e42" path="/var/lib/kubelet/pods/5ba2ecac-51e9-491a-a01b-fc8e74ab6e42/volumes"
Sep 10 19:41:05.891429 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.891386    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:05.897106 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:05.897074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:06.109179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:06.109148    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:06.110278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:06.110257    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:06.282063 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:06.282035    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:06.370711 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:06.370609    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fphn2\" (UniqueName: \"kubernetes.io/projected/da0664e4-56c6-48ca-8eb4-dcce53b06606-kube-api-access-fphn2\") pod \"da0664e4-56c6-48ca-8eb4-dcce53b06606\" (UID: \"da0664e4-56c6-48ca-8eb4-dcce53b06606\") "
Sep 10 19:41:06.372854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:06.372826    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da0664e4-56c6-48ca-8eb4-dcce53b06606-kube-api-access-fphn2" (OuterVolumeSpecName: "kube-api-access-fphn2") pod "da0664e4-56c6-48ca-8eb4-dcce53b06606" (UID: "da0664e4-56c6-48ca-8eb4-dcce53b06606"). InnerVolumeSpecName "kube-api-access-fphn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:06.478348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:06.473112    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fphn2\" (UniqueName: \"kubernetes.io/projected/da0664e4-56c6-48ca-8eb4-dcce53b06606-kube-api-access-fphn2\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:41:06.912809 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:06.912767    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:41:07.104250 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.104189    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:07.116912 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.116843    2562 generic.go:356] "Generic (PLEG): container finished" podID="da0664e4-56c6-48ca-8eb4-dcce53b06606" containerID="5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc" exitCode=137
Sep 10 19:41:07.117337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.116920    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:07.117337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.116941    2562 scope.go:117] "RemoveContainer" containerID="5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc"
Sep 10 19:41:07.128637 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.128434    2562 scope.go:117] "RemoveContainer" containerID="5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc"
Sep 10 19:41:07.128637 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:07.128506    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7733-fad49d\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="da0664e4-56c6-48ca-8eb4-dcce53b06606" pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:07.128807 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:07.128785    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc\": container with ID starting with 5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc not found: ID does not exist" containerID="5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc"
Sep 10 19:41:07.128861 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.128816    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc"} err="failed to get container status \"5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc\": rpc error: code = NotFound desc = could not find container \"5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc\": container with ID starting with 5d1ce4ad758b816364eda8392cd649eaf3087cc3e8608346d6fbf3b1351d77bc not found: ID does not exist"
Sep 10 19:41:07.197856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.197743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:07.694731 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:41:07.694692    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7733-fad49d\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="da0664e4-56c6-48ca-8eb4-dcce53b06606" pod="opendatahub/test-curl-7733-fad49d"
Sep 10 19:41:07.698280 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.698251    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da0664e4-56c6-48ca-8eb4-dcce53b06606" path="/var/lib/kubelet/pods/da0664e4-56c6-48ca-8eb4-dcce53b06606/volumes"
Sep 10 19:41:07.861668 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.861633    2562 scope.go:117] "RemoveContainer" containerID="28bd2bcc29484acdee5e0c7bde9c94e9751ad2e27918ab2e8b60b4fddbeb36fc"
Sep 10 19:41:07.870517 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:07.870494    2562 scope.go:117] "RemoveContainer" containerID="c248f3b3eefed59cb2d3485ea71aab95703621868219c5677c9d69583a607ef3"
Sep 10 19:41:08.322820 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:08.322775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:08.425656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:08.425618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:08.695039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:08.695000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:08.835291 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:08.835250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:08.944416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:08.944363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:08.993970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:08.993873    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"]
Sep 10 19:41:08.998750 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:08.998730    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.001651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.001625    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b97a9c60-dockercfg-s7zfr\""
Sep 10 19:41:09.001909 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.001880    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b97a9c60\""
Sep 10 19:41:09.019135 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.019099    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"]
Sep 10 19:41:09.046328 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.046294    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"]
Sep 10 19:41:09.049880 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.049854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.061023 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.060991    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"]
Sep 10 19:41:09.101372 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.101333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b88zn\" (UniqueName: \"kubernetes.io/projected/4827813a-12bd-474c-9292-f8cb181889fc-kube-api-access-b88zn\") pod \"payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") " pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.101583 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.101474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c916c2-c3b8-4b49-a126-7db3329c73fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.101583 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.101552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjdrg\" (UniqueName: \"kubernetes.io/projected/32c916c2-c3b8-4b49-a126-7db3329c73fd-kube-api-access-xjdrg\") pod \"payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.101695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.101591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4827813a-12bd-474c-9292-f8cb181889fc-plugins-config-volume\") pod \"payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") " pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.202527 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.202483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c916c2-c3b8-4b49-a126-7db3329c73fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.202738 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.202551    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjdrg\" (UniqueName: \"kubernetes.io/projected/32c916c2-c3b8-4b49-a126-7db3329c73fd-kube-api-access-xjdrg\") pod \"payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.202738 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.202577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4827813a-12bd-474c-9292-f8cb181889fc-plugins-config-volume\") pod \"payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") " pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.202738 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.202599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b88zn\" (UniqueName: \"kubernetes.io/projected/4827813a-12bd-474c-9292-f8cb181889fc-kube-api-access-b88zn\") pod \"payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") " pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.203334 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.203309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c916c2-c3b8-4b49-a126-7db3329c73fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.203421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.203340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4827813a-12bd-474c-9292-f8cb181889fc-plugins-config-volume\") pod \"payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") " pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.211265 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.211227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b88zn\" (UniqueName: \"kubernetes.io/projected/4827813a-12bd-474c-9292-f8cb181889fc-kube-api-access-b88zn\") pod \"payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") " pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.211480 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.211460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjdrg\" (UniqueName: \"kubernetes.io/projected/32c916c2-c3b8-4b49-a126-7db3329c73fd-kube-api-access-xjdrg\") pod \"payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.309255 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.309157    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:09.361285 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.361182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:09.598312 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.598282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"]
Sep 10 19:41:09.600186 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:41:09.600149    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4827813a_12bd_474c_9292_f8cb181889fc.slice/crio-bcb0da46f107f886346291d991b2fcc8b3e97a02c6f845e61d15876ed041b88b WatchSource:0}: Error finding container bcb0da46f107f886346291d991b2fcc8b3e97a02c6f845e61d15876ed041b88b: Status 404 returned error can't find the container with id bcb0da46f107f886346291d991b2fcc8b3e97a02c6f845e61d15876ed041b88b
Sep 10 19:41:09.647389 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:09.647172    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"]
Sep 10 19:41:09.649846 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:41:09.649816    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c916c2_c3b8_4b49_a126_7db3329c73fd.slice/crio-a920b699b37d7c422896767a63e72238cde7d4104bbe1be3a476c61f8459545b WatchSource:0}: Error finding container a920b699b37d7c422896767a63e72238cde7d4104bbe1be3a476c61f8459545b: Status 404 returned error can't find the container with id a920b699b37d7c422896767a63e72238cde7d4104bbe1be3a476c61f8459545b
Sep 10 19:41:10.135015 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.134978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx" event={"ID":"32c916c2-c3b8-4b49-a126-7db3329c73fd","Type":"ContainerStarted","Data":"1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729"}
Sep 10 19:41:10.135213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.135043    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx" event={"ID":"32c916c2-c3b8-4b49-a126-7db3329c73fd","Type":"ContainerStarted","Data":"a920b699b37d7c422896767a63e72238cde7d4104bbe1be3a476c61f8459545b"}
Sep 10 19:41:10.135213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.135074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:10.136701 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.136671    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt" event={"ID":"4827813a-12bd-474c-9292-f8cb181889fc","Type":"ContainerStarted","Data":"f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332"}
Sep 10 19:41:10.136823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.136723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt" event={"ID":"4827813a-12bd-474c-9292-f8cb181889fc","Type":"ContainerStarted","Data":"bcb0da46f107f886346291d991b2fcc8b3e97a02c6f845e61d15876ed041b88b"}
Sep 10 19:41:10.136823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.136762    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:10.154332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.154273    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx" podStartSLOduration=1.154257001 podStartE2EDuration="1.154257001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:10.152512669 +0000 UTC m=+1263.092546503" watchObservedRunningTime="2026-09-10 19:41:10.154257001 +0000 UTC m=+1263.094290833"
Sep 10 19:41:10.173134 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:10.173061    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt" podStartSLOduration=2.173039967 podStartE2EDuration="2.173039967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:10.171391652 +0000 UTC m=+1263.111425484" watchObservedRunningTime="2026-09-10 19:41:10.173039967 +0000 UTC m=+1263.113073803"
Sep 10 19:41:21.147800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:21.147757    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:41:21.148699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:21.148671    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:41:23.301442 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:23.301407    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:41:27.232777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:27.232685    2562 generic.go:356] "Generic (PLEG): container finished" podID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerID="1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12" exitCode=0
Sep 10 19:41:27.232777 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:27.232754    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" event={"ID":"949ec5a0-f9ac-47a4-ab61-3f857c94ca41","Type":"ContainerDied","Data":"1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12"}
Sep 10 19:41:27.233241 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:27.233222    2562 scope.go:117] "RemoveContainer" containerID="1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12"
Sep 10 19:41:27.234349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:27.234319    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:41:27.234414 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:27.234392    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:41:27.234451 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:27.234426    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:41:28.240687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:28.240650    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" event={"ID":"949ec5a0-f9ac-47a4-ab61-3f857c94ca41","Type":"ContainerStarted","Data":"d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7"}
Sep 10 19:41:28.890880 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:28.890837    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:28.891295 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:28.891266    2562 patch_prober.go:28] interesting pod/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:28.891411 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:28.891339    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:29.891953 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:29.891881    2562 patch_prober.go:28] interesting pod/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:29.892384 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:29.891981    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:30.891725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:30.891681    2562 patch_prober.go:28] interesting pod/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:30.891942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:30.891745    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:31.891633 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:31.891582    2562 patch_prober.go:28] interesting pod/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:31.892172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:31.891655    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:31.980585 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:31.980552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:32.891601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:32.891555    2562 patch_prober.go:28] interesting pod/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:32.891824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:32.891623    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:33.891151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:33.891115    2562 patch_prober.go:28] interesting pod/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 10 19:41:33.891337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:33.891171    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 10 19:41:34.896533 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:34.896496    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:34.897200 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:34.897171    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:34.898392 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:34.898371    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:41:35.898246 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:35.898211    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:41:40.418979 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:40.418943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:54.010181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:54.010138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:41:55.710238 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:41:55.710196    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:42:07.910310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:07.910271    2562 scope.go:117] "RemoveContainer" containerID="8781762deba6f43c37dbe485dab55145c750784292fd5a5cd15d0a4e8f3dac27"
Sep 10 19:42:07.919446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:07.919422    2562 scope.go:117] "RemoveContainer" containerID="fe25e12abc8cf2b0db7ff6cd9f177b06c16abee241d67ea47a609d2f601c5cd7"
Sep 10 19:42:14.898832 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:14.898794    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:42:21.639279 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.639237    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"]
Sep 10 19:42:21.644385 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.644364    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"
Sep 10 19:42:21.647163 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.647110    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:42:21.647966 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.647917    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:42:21.649469 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.649443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"]
Sep 10 19:42:21.721253 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.721207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f697a6a-7b39-4533-8be4-e11509f38329-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv\" (UID: \"6f697a6a-7b39-4533-8be4-e11509f38329\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"
Sep 10 19:42:21.749038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.748999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:21.822369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.822328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f697a6a-7b39-4533-8be4-e11509f38329-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv\" (UID: \"6f697a6a-7b39-4533-8be4-e11509f38329\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"
Sep 10 19:42:21.823008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.822988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f697a6a-7b39-4533-8be4-e11509f38329-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv\" (UID: \"6f697a6a-7b39-4533-8be4-e11509f38329\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"
Sep 10 19:42:21.958507 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:21.958410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"
Sep 10 19:42:22.232198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:22.232173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"]
Sep 10 19:42:22.234383 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:42:22.234355    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f697a6a_7b39_4533_8be4_e11509f38329.slice/crio-0a22a5b0dc5ab0156e0e8a5b503c92f3e436b4f5966fa5f35f3be0687f2bfa47 WatchSource:0}: Error finding container 0a22a5b0dc5ab0156e0e8a5b503c92f3e436b4f5966fa5f35f3be0687f2bfa47: Status 404 returned error can't find the container with id 0a22a5b0dc5ab0156e0e8a5b503c92f3e436b4f5966fa5f35f3be0687f2bfa47
Sep 10 19:42:22.564425 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:22.564390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv" event={"ID":"6f697a6a-7b39-4533-8be4-e11509f38329","Type":"ContainerStarted","Data":"e9b64736c247d59a6508e76a38e0c4477ebe440b7608a56f70ede1685852e2db"}
Sep 10 19:42:22.564807 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:22.564446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv" event={"ID":"6f697a6a-7b39-4533-8be4-e11509f38329","Type":"ContainerStarted","Data":"0a22a5b0dc5ab0156e0e8a5b503c92f3e436b4f5966fa5f35f3be0687f2bfa47"}
Sep 10 19:42:22.583174 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:22.583112    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv" podStartSLOduration=1.5830934939999999 podStartE2EDuration="1.583093494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:22.581249743 +0000 UTC m=+1335.521283576" watchObservedRunningTime="2026-09-10 19:42:22.583093494 +0000 UTC m=+1335.523127327"
Sep 10 19:42:23.571571 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:23.571533    2562 generic.go:356] "Generic (PLEG): container finished" podID="6f697a6a-7b39-4533-8be4-e11509f38329" containerID="e9b64736c247d59a6508e76a38e0c4477ebe440b7608a56f70ede1685852e2db" exitCode=0
Sep 10 19:42:23.571976 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:23.571623    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv" event={"ID":"6f697a6a-7b39-4533-8be4-e11509f38329","Type":"ContainerDied","Data":"e9b64736c247d59a6508e76a38e0c4477ebe440b7608a56f70ede1685852e2db"}
Sep 10 19:42:24.710402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:24.710372    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"
Sep 10 19:42:24.750826 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:24.750787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f697a6a-7b39-4533-8be4-e11509f38329-maas-api-service-ca\") pod \"6f697a6a-7b39-4533-8be4-e11509f38329\" (UID: \"6f697a6a-7b39-4533-8be4-e11509f38329\") "
Sep 10 19:42:24.751260 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:24.751229    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f697a6a-7b39-4533-8be4-e11509f38329-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f697a6a-7b39-4533-8be4-e11509f38329" (UID: "6f697a6a-7b39-4533-8be4-e11509f38329"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:24.751372 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:24.751338    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f697a6a-7b39-4533-8be4-e11509f38329-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:42:25.583629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:25.583601    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"
Sep 10 19:42:25.583629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:25.583614    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv" event={"ID":"6f697a6a-7b39-4533-8be4-e11509f38329","Type":"ContainerDied","Data":"0a22a5b0dc5ab0156e0e8a5b503c92f3e436b4f5966fa5f35f3be0687f2bfa47"}
Sep 10 19:42:25.583816 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:25.583664    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a22a5b0dc5ab0156e0e8a5b503c92f3e436b4f5966fa5f35f3be0687f2bfa47"
Sep 10 19:42:33.404433 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:33.404340    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:42:37.216726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.216678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:37.256368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.256322    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"]
Sep 10 19:42:37.256651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.256613    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt" podUID="4827813a-12bd-474c-9292-f8cb181889fc" containerName="payload-processing" containerID="cri-o://f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332" gracePeriod=30
Sep 10 19:42:37.278981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.278952    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"]
Sep 10 19:42:37.279179 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.279157    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx" podUID="32c916c2-c3b8-4b49-a126-7db3329c73fd" containerName="payload-pre-processing" containerID="cri-o://1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729" gracePeriod=30
Sep 10 19:42:37.317689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.317652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:37.418264 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.418164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:37.492300 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.492124    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"]
Sep 10 19:42:37.494584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.494554    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b97a9c60-jbtlv"]
Sep 10 19:42:37.499620 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.499559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:37.550156 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.550129    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:42:37.581004 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.580975    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:42:37.653120 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.653084    2562 generic.go:356] "Generic (PLEG): container finished" podID="4827813a-12bd-474c-9292-f8cb181889fc" containerID="f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332" exitCode=0
Sep 10 19:42:37.653315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.653132    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"
Sep 10 19:42:37.653315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.653164    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt" event={"ID":"4827813a-12bd-474c-9292-f8cb181889fc","Type":"ContainerDied","Data":"f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332"}
Sep 10 19:42:37.653315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.653217    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt" event={"ID":"4827813a-12bd-474c-9292-f8cb181889fc","Type":"ContainerDied","Data":"bcb0da46f107f886346291d991b2fcc8b3e97a02c6f845e61d15876ed041b88b"}
Sep 10 19:42:37.653315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.653240    2562 scope.go:117] "RemoveContainer" containerID="f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332"
Sep 10 19:42:37.654608 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.654588    2562 generic.go:356] "Generic (PLEG): container finished" podID="32c916c2-c3b8-4b49-a126-7db3329c73fd" containerID="1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729" exitCode=0
Sep 10 19:42:37.654722 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.654633    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"
Sep 10 19:42:37.654722 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.654674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx" event={"ID":"32c916c2-c3b8-4b49-a126-7db3329c73fd","Type":"ContainerDied","Data":"1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729"}
Sep 10 19:42:37.654810 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.654727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx" event={"ID":"32c916c2-c3b8-4b49-a126-7db3329c73fd","Type":"ContainerDied","Data":"a920b699b37d7c422896767a63e72238cde7d4104bbe1be3a476c61f8459545b"}
Sep 10 19:42:37.662542 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.662513    2562 scope.go:117] "RemoveContainer" containerID="f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332"
Sep 10 19:42:37.662857 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:42:37.662837    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332\": container with ID starting with f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332 not found: ID does not exist" containerID="f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332"
Sep 10 19:42:37.662989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.662866    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332"} err="failed to get container status \"f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332\": rpc error: code = NotFound desc = could not find container \"f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332\": container with ID starting with f08d844036db6cf658c3edab3cea55058fb9cc809dbb809a4638f09baeb82332 not found: ID does not exist"
Sep 10 19:42:37.662989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.662898    2562 scope.go:117] "RemoveContainer" containerID="1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729"
Sep 10 19:42:37.671167 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.671140    2562 scope.go:117] "RemoveContainer" containerID="1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729"
Sep 10 19:42:37.671496 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:42:37.671476    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729\": container with ID starting with 1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729 not found: ID does not exist" containerID="1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729"
Sep 10 19:42:37.671562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.671507    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729"} err="failed to get container status \"1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729\": rpc error: code = NotFound desc = could not find container \"1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729\": container with ID starting with 1d169076eb2a92d0c1052e6c2160baf30f5f4f60c349d995826cde176d8a7729 not found: ID does not exist"
Sep 10 19:42:37.677831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.677806    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b88zn\" (UniqueName: \"kubernetes.io/projected/4827813a-12bd-474c-9292-f8cb181889fc-kube-api-access-b88zn\") pod \"4827813a-12bd-474c-9292-f8cb181889fc\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") "
Sep 10 19:42:37.677962 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.677854    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4827813a-12bd-474c-9292-f8cb181889fc-plugins-config-volume\") pod \"4827813a-12bd-474c-9292-f8cb181889fc\" (UID: \"4827813a-12bd-474c-9292-f8cb181889fc\") "
Sep 10 19:42:37.678020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.678007    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjdrg\" (UniqueName: \"kubernetes.io/projected/32c916c2-c3b8-4b49-a126-7db3329c73fd-kube-api-access-xjdrg\") pod \"32c916c2-c3b8-4b49-a126-7db3329c73fd\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") "
Sep 10 19:42:37.678062 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.678049    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c916c2-c3b8-4b49-a126-7db3329c73fd-plugins-config-volume\") pod \"32c916c2-c3b8-4b49-a126-7db3329c73fd\" (UID: \"32c916c2-c3b8-4b49-a126-7db3329c73fd\") "
Sep 10 19:42:37.678186 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.678165    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4827813a-12bd-474c-9292-f8cb181889fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4827813a-12bd-474c-9292-f8cb181889fc" (UID: "4827813a-12bd-474c-9292-f8cb181889fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:37.678434 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.678412    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4827813a-12bd-474c-9292-f8cb181889fc-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:42:37.678496 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.678437    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c916c2-c3b8-4b49-a126-7db3329c73fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32c916c2-c3b8-4b49-a126-7db3329c73fd" (UID: "32c916c2-c3b8-4b49-a126-7db3329c73fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:37.680020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.680001    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c916c2-c3b8-4b49-a126-7db3329c73fd-kube-api-access-xjdrg" (OuterVolumeSpecName: "kube-api-access-xjdrg") pod "32c916c2-c3b8-4b49-a126-7db3329c73fd" (UID: "32c916c2-c3b8-4b49-a126-7db3329c73fd"). InnerVolumeSpecName "kube-api-access-xjdrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:37.680020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.680011    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4827813a-12bd-474c-9292-f8cb181889fc-kube-api-access-b88zn" (OuterVolumeSpecName: "kube-api-access-b88zn") pod "4827813a-12bd-474c-9292-f8cb181889fc" (UID: "4827813a-12bd-474c-9292-f8cb181889fc"). InnerVolumeSpecName "kube-api-access-b88zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:37.696108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.696076    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f697a6a-7b39-4533-8be4-e11509f38329" path="/var/lib/kubelet/pods/6f697a6a-7b39-4533-8be4-e11509f38329/volumes"
Sep 10 19:42:37.779081 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.778972    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b88zn\" (UniqueName: \"kubernetes.io/projected/4827813a-12bd-474c-9292-f8cb181889fc-kube-api-access-b88zn\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:42:37.779081 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.779010    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjdrg\" (UniqueName: \"kubernetes.io/projected/32c916c2-c3b8-4b49-a126-7db3329c73fd-kube-api-access-xjdrg\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:42:37.779081 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.779023    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32c916c2-c3b8-4b49-a126-7db3329c73fd-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:42:37.969775 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.969737    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"]
Sep 10 19:42:37.976233 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.976195    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b97a9c60-854fc8cb69-476dt"]
Sep 10 19:42:37.986633 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.986599    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"]
Sep 10 19:42:37.990832 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:37.990798    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b97a9c60-54654d5dc8-hbjbx"]
Sep 10 19:42:39.696486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:39.696451    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32c916c2-c3b8-4b49-a126-7db3329c73fd" path="/var/lib/kubelet/pods/32c916c2-c3b8-4b49-a126-7db3329c73fd/volumes"
Sep 10 19:42:39.696984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:39.696838    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4827813a-12bd-474c-9292-f8cb181889fc" path="/var/lib/kubelet/pods/4827813a-12bd-474c-9292-f8cb181889fc/volumes"
Sep 10 19:42:40.673805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:40.673768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:45.696922 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:45.696871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:42:52.295699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:42:52.295654    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:43:07.944211 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:07.944176    2562 scope.go:117] "RemoveContainer" containerID="f3b731c03556b628c293cf07c095a46f9aafe70b5c55e3c507cbb4e32f6ec355"
Sep 10 19:43:08.808133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:08.808092    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:43:11.887600 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.887562    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7736-8b0f68"]
Sep 10 19:43:11.888097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.887991    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4827813a-12bd-474c-9292-f8cb181889fc" containerName="payload-processing"
Sep 10 19:43:11.888097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888005    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4827813a-12bd-474c-9292-f8cb181889fc" containerName="payload-processing"
Sep 10 19:43:11.888097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888026    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c916c2-c3b8-4b49-a126-7db3329c73fd" containerName="payload-pre-processing"
Sep 10 19:43:11.888097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888032    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c916c2-c3b8-4b49-a126-7db3329c73fd" containerName="payload-pre-processing"
Sep 10 19:43:11.888097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888057    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f697a6a-7b39-4533-8be4-e11509f38329" containerName="revoke-keys"
Sep 10 19:43:11.888097 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888063    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f697a6a-7b39-4533-8be4-e11509f38329" containerName="revoke-keys"
Sep 10 19:43:11.888298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888128    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f697a6a-7b39-4533-8be4-e11509f38329" containerName="revoke-keys"
Sep 10 19:43:11.888298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888140    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c916c2-c3b8-4b49-a126-7db3329c73fd" containerName="payload-pre-processing"
Sep 10 19:43:11.888298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.888148    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4827813a-12bd-474c-9292-f8cb181889fc" containerName="payload-processing"
Sep 10 19:43:11.892629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.892609    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-8b0f68"
Sep 10 19:43:11.895619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.895591    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zp7h9\""
Sep 10 19:43:11.895813 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:11.895713    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7736-8b0f68"]
Sep 10 19:43:12.014110 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.014076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2w8j\" (UniqueName: \"kubernetes.io/projected/d9d02a35-178b-4a51-a4e4-d502e2e00327-kube-api-access-f2w8j\") pod \"test-curl-7736-8b0f68\" (UID: \"d9d02a35-178b-4a51-a4e4-d502e2e00327\") " pod="opendatahub/test-curl-7736-8b0f68"
Sep 10 19:43:12.115590 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.115526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2w8j\" (UniqueName: \"kubernetes.io/projected/d9d02a35-178b-4a51-a4e4-d502e2e00327-kube-api-access-f2w8j\") pod \"test-curl-7736-8b0f68\" (UID: \"d9d02a35-178b-4a51-a4e4-d502e2e00327\") " pod="opendatahub/test-curl-7736-8b0f68"
Sep 10 19:43:12.125558 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.125529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2w8j\" (UniqueName: \"kubernetes.io/projected/d9d02a35-178b-4a51-a4e4-d502e2e00327-kube-api-access-f2w8j\") pod \"test-curl-7736-8b0f68\" (UID: \"d9d02a35-178b-4a51-a4e4-d502e2e00327\") " pod="opendatahub/test-curl-7736-8b0f68"
Sep 10 19:43:12.203584 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.203543    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-8b0f68"
Sep 10 19:43:12.473121 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.473096    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7736-8b0f68"]
Sep 10 19:43:12.475327 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:43:12.475298    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9d02a35_178b_4a51_a4e4_d502e2e00327.slice/crio-3f9263c83fe3f10291f8c085da3696198c37b0caea551e1d4aaceb92f3e1b95c WatchSource:0}: Error finding container 3f9263c83fe3f10291f8c085da3696198c37b0caea551e1d4aaceb92f3e1b95c: Status 404 returned error can't find the container with id 3f9263c83fe3f10291f8c085da3696198c37b0caea551e1d4aaceb92f3e1b95c
Sep 10 19:43:12.847927 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.847814    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7736-8b0f68" event={"ID":"d9d02a35-178b-4a51-a4e4-d502e2e00327","Type":"ContainerStarted","Data":"5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083"}
Sep 10 19:43:12.847927 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.847860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7736-8b0f68" event={"ID":"d9d02a35-178b-4a51-a4e4-d502e2e00327","Type":"ContainerStarted","Data":"3f9263c83fe3f10291f8c085da3696198c37b0caea551e1d4aaceb92f3e1b95c"}
Sep 10 19:43:12.866143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:12.866087    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7736-8b0f68" podStartSLOduration=1.866068644 podStartE2EDuration="1.866068644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:43:12.864858142 +0000 UTC m=+1385.804891976" watchObservedRunningTime="2026-09-10 19:43:12.866068644 +0000 UTC m=+1385.806102513"
Sep 10 19:43:13.498807 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:13.498767    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7736-8b0f68"]
Sep 10 19:43:13.503496 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:13.503457    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7736-8b0f68"]
Sep 10 19:43:13.852919 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:43:13.852876    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7736-8b0f68" container="test-curl-7736-8b0f68"
Sep 10 19:43:14.858082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:14.858044    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7736-8b0f68" podUID="d9d02a35-178b-4a51-a4e4-d502e2e00327" containerName="test-curl-7736-8b0f68" containerID="cri-o://5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083" gracePeriod=2
Sep 10 19:43:15.058504 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:15.058468    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"]
Sep 10 19:43:15.088399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:15.088365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:16.119709 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:16.119662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:17.048616 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.048590    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-8b0f68"
Sep 10 19:43:17.164845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.164805    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2w8j\" (UniqueName: \"kubernetes.io/projected/d9d02a35-178b-4a51-a4e4-d502e2e00327-kube-api-access-f2w8j\") pod \"d9d02a35-178b-4a51-a4e4-d502e2e00327\" (UID: \"d9d02a35-178b-4a51-a4e4-d502e2e00327\") "
Sep 10 19:43:17.167042 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.167012    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d02a35-178b-4a51-a4e4-d502e2e00327-kube-api-access-f2w8j" (OuterVolumeSpecName: "kube-api-access-f2w8j") pod "d9d02a35-178b-4a51-a4e4-d502e2e00327" (UID: "d9d02a35-178b-4a51-a4e4-d502e2e00327"). InnerVolumeSpecName "kube-api-access-f2w8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:17.265981 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.265941    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2w8j\" (UniqueName: \"kubernetes.io/projected/d9d02a35-178b-4a51-a4e4-d502e2e00327-kube-api-access-f2w8j\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:43:17.695924 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.695872    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9d02a35-178b-4a51-a4e4-d502e2e00327" path="/var/lib/kubelet/pods/d9d02a35-178b-4a51-a4e4-d502e2e00327/volumes"
Sep 10 19:43:17.876288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.876246    2562 generic.go:356] "Generic (PLEG): container finished" podID="d9d02a35-178b-4a51-a4e4-d502e2e00327" containerID="5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083" exitCode=137
Sep 10 19:43:17.876288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.876287    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-8b0f68"
Sep 10 19:43:17.876524 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.876357    2562 scope.go:117] "RemoveContainer" containerID="5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083"
Sep 10 19:43:17.887685 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.887662    2562 scope.go:117] "RemoveContainer" containerID="5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083"
Sep 10 19:43:17.888055 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:43:17.888034    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083\": container with ID starting with 5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083 not found: ID does not exist" containerID="5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083"
Sep 10 19:43:17.888127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:17.888067    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083"} err="failed to get container status \"5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083\": rpc error: code = NotFound desc = could not find container \"5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083\": container with ID starting with 5977238bbf83868b75150c8c2c1b43dc24cc6871343ae2461177a18a72036083 not found: ID does not exist"
Sep 10 19:43:20.000350 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:20.000312    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:43:23.047451 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:23.047415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:23.836643 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:23.836610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:30.079287 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.079222    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm" podUID="58433946-c553-4b13-968e-6b769fa4e936" containerName="main" containerID="cri-o://b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65" gracePeriod=45
Sep 10 19:43:30.329178 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.329149    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:43:30.487272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487236    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-dshm\") pod \"58433946-c553-4b13-968e-6b769fa4e936\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") "
Sep 10 19:43:30.487479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487292    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-model-cache\") pod \"58433946-c553-4b13-968e-6b769fa4e936\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") "
Sep 10 19:43:30.487479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487348    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58433946-c553-4b13-968e-6b769fa4e936-tls-certs\") pod \"58433946-c553-4b13-968e-6b769fa4e936\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") "
Sep 10 19:43:30.487479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487424    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rht2l\" (UniqueName: \"kubernetes.io/projected/58433946-c553-4b13-968e-6b769fa4e936-kube-api-access-rht2l\") pod \"58433946-c553-4b13-968e-6b769fa4e936\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") "
Sep 10 19:43:30.487479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487475    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-tmp-dir\") pod \"58433946-c553-4b13-968e-6b769fa4e936\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") "
Sep 10 19:43:30.487629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487518    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-home\") pod \"58433946-c553-4b13-968e-6b769fa4e936\" (UID: \"58433946-c553-4b13-968e-6b769fa4e936\") "
Sep 10 19:43:30.487677 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487625    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-model-cache" (OuterVolumeSpecName: "model-cache") pod "58433946-c553-4b13-968e-6b769fa4e936" (UID: "58433946-c553-4b13-968e-6b769fa4e936"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:30.487836 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487801    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "58433946-c553-4b13-968e-6b769fa4e936" (UID: "58433946-c553-4b13-968e-6b769fa4e936"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:30.487836 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487818    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-home" (OuterVolumeSpecName: "home") pod "58433946-c553-4b13-968e-6b769fa4e936" (UID: "58433946-c553-4b13-968e-6b769fa4e936"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:30.488030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.487872    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:43:30.489604 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.489579    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-dshm" (OuterVolumeSpecName: "dshm") pod "58433946-c553-4b13-968e-6b769fa4e936" (UID: "58433946-c553-4b13-968e-6b769fa4e936"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:30.489979 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.489961    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58433946-c553-4b13-968e-6b769fa4e936-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "58433946-c553-4b13-968e-6b769fa4e936" (UID: "58433946-c553-4b13-968e-6b769fa4e936"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:43:30.490030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.489989    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58433946-c553-4b13-968e-6b769fa4e936-kube-api-access-rht2l" (OuterVolumeSpecName: "kube-api-access-rht2l") pod "58433946-c553-4b13-968e-6b769fa4e936" (UID: "58433946-c553-4b13-968e-6b769fa4e936"). InnerVolumeSpecName "kube-api-access-rht2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:30.588853 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.588807    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rht2l\" (UniqueName: \"kubernetes.io/projected/58433946-c553-4b13-968e-6b769fa4e936-kube-api-access-rht2l\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:43:30.588853 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.588846    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:43:30.588853 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.588856    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:43:30.588853 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.588864    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58433946-c553-4b13-968e-6b769fa4e936-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:43:30.589334 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.588872    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58433946-c553-4b13-968e-6b769fa4e936-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:43:30.945852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.945820    2562 generic.go:356] "Generic (PLEG): container finished" podID="58433946-c553-4b13-968e-6b769fa4e936" containerID="b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65" exitCode=0
Sep 10 19:43:30.946034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.945862    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"
Sep 10 19:43:30.946034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.945921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm" event={"ID":"58433946-c553-4b13-968e-6b769fa4e936","Type":"ContainerDied","Data":"b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65"}
Sep 10 19:43:30.946034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.945971    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm" event={"ID":"58433946-c553-4b13-968e-6b769fa4e936","Type":"ContainerDied","Data":"f165d6583a733e4b9d4485564df81c492f395842552dcf43cc8d075d4e3b71af"}
Sep 10 19:43:30.946034 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.945990    2562 scope.go:117] "RemoveContainer" containerID="b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65"
Sep 10 19:43:30.955646 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.955623    2562 scope.go:117] "RemoveContainer" containerID="b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65"
Sep 10 19:43:30.955960 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:43:30.955940    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65\": container with ID starting with b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65 not found: ID does not exist" containerID="b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65"
Sep 10 19:43:30.956067 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.955966    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65"} err="failed to get container status \"b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65\": rpc error: code = NotFound desc = could not find container \"b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65\": container with ID starting with b6f60baa7be474fad8ceac392620b08d7c9c45af4115e4c7fc8e52b85eeaaf65 not found: ID does not exist"
Sep 10 19:43:30.969138 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.969107    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"]
Sep 10 19:43:30.973817 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:30.973789    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/auth-test-model-07c5fe-kserve-74458dbfdc-t9prm"]
Sep 10 19:43:31.695491 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:31.695455    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58433946-c553-4b13-968e-6b769fa4e936" path="/var/lib/kubelet/pods/58433946-c553-4b13-968e-6b769fa4e936/volumes"
Sep 10 19:43:46.704679 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.704645    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"]
Sep 10 19:43:46.705175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.705070    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9d02a35-178b-4a51-a4e4-d502e2e00327" containerName="test-curl-7736-8b0f68"
Sep 10 19:43:46.705175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.705085    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9d02a35-178b-4a51-a4e4-d502e2e00327" containerName="test-curl-7736-8b0f68"
Sep 10 19:43:46.705175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.705106    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58433946-c553-4b13-968e-6b769fa4e936" containerName="main"
Sep 10 19:43:46.705175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.705111    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="58433946-c553-4b13-968e-6b769fa4e936" containerName="main"
Sep 10 19:43:46.705175 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.705174    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="58433946-c553-4b13-968e-6b769fa4e936" containerName="main"
Sep 10 19:43:46.705357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.705182    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9d02a35-178b-4a51-a4e4-d502e2e00327" containerName="test-curl-7736-8b0f68"
Sep 10 19:43:46.708298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.708280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.711250 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.711225    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"kube-root-ca.crt\""
Sep 10 19:43:46.712236 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.712210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"openshift-service-ca.crt\""
Sep 10 19:43:46.712346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.712255    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"sub-test-model-07c5fe-kserve-self-signed-certs\""
Sep 10 19:43:46.712346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.712298    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"default-dockercfg-6ktvk\""
Sep 10 19:43:46.723327 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.723298    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"]
Sep 10 19:43:46.733527 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.733481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9w86\" (UniqueName: \"kubernetes.io/projected/9e88bf98-08fb-4af9-b741-d129d2aa0006-kube-api-access-b9w86\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.733718 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.733586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-tmp-dir\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.733718 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.733667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e88bf98-08fb-4af9-b741-d129d2aa0006-tls-certs\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.733833 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.733743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-dshm\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.733833 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.733791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-model-cache\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.733962 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.733836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-home\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.754790 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.754753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:46.835309 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835267    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9w86\" (UniqueName: \"kubernetes.io/projected/9e88bf98-08fb-4af9-b741-d129d2aa0006-kube-api-access-b9w86\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-tmp-dir\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835361    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e88bf98-08fb-4af9-b741-d129d2aa0006-tls-certs\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-dshm\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-model-cache\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-home\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835808 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-tmp-dir\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-home\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.835954 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.835879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-model-cache\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.837735 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.837710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-dshm\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.838059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.838042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e88bf98-08fb-4af9-b741-d129d2aa0006-tls-certs\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:46.845992 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:46.845961    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9w86\" (UniqueName: \"kubernetes.io/projected/9e88bf98-08fb-4af9-b741-d129d2aa0006-kube-api-access-b9w86\") pod \"sub-test-model-07c5fe-kserve-595b79ffbc-fvllc\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:47.018256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:47.018158    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:47.301069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:47.300970    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"]
Sep 10 19:43:47.304059 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:43:47.304026    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e88bf98_08fb_4af9_b741_d129d2aa0006.slice/crio-41518f168a1d865fc45140c739233a487a6664a6087492023c98ee299ef5bf45 WatchSource:0}: Error finding container 41518f168a1d865fc45140c739233a487a6664a6087492023c98ee299ef5bf45: Status 404 returned error can't find the container with id 41518f168a1d865fc45140c739233a487a6664a6087492023c98ee299ef5bf45
Sep 10 19:43:48.039509 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:48.039475    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc" event={"ID":"9e88bf98-08fb-4af9-b741-d129d2aa0006","Type":"ContainerStarted","Data":"5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac"}
Sep 10 19:43:48.039875 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:48.039519    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc" event={"ID":"9e88bf98-08fb-4af9-b741-d129d2aa0006","Type":"ContainerStarted","Data":"41518f168a1d865fc45140c739233a487a6664a6087492023c98ee299ef5bf45"}
Sep 10 19:43:48.060039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:48.059975    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc" podStartSLOduration=2.059958739 podStartE2EDuration="2.059958739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:43:48.058031997 +0000 UTC m=+1420.998065824" watchObservedRunningTime="2026-09-10 19:43:48.059958739 +0000 UTC m=+1420.999992622"
Sep 10 19:43:50.711144 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:50.711093    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:43:57.018494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.018452    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:57.018979 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.018511    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:57.031450 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.031418    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:57.098788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.098750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:57.100492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.100466    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:43:57.198082 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.198027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:57.396815 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.396777    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"]
Sep 10 19:43:57.397052 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.397030    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy" containerID="cri-o://d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7" gracePeriod=30
Sep 10 19:43:57.440596 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:57.440551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:58.678246 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:58.678203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:58.797718 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:58.797670    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:59.521456 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:59.521415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:43:59.626560 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:43:59.626515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:00.987314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:00.987271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:02.214980 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.214943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:02.653334 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.653307    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:44:02.763960 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.763918    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:44:02.791488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791398    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-token\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791446    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtff2\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-kube-api-access-rtff2\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791488 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791487    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-credential-socket\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791507    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-data\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791535    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istiod-ca-cert\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791586    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-certs\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791619    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-podinfo\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791761    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-socket\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791793    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-envoy\") pod \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\" (UID: \"949ec5a0-f9ac-47a4-ab61-3f857c94ca41\") "
Sep 10 19:44:02.791840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.791791    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:02.792273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.792045    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-data" (OuterVolumeSpecName: "istio-data") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:02.792273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.792071    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:02.792273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.792144    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:02.792273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.792189    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:02.793678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.793518    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.793678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.793579    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-workload-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.793678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.793594    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-credential-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.793678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.793611    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-data\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.793678 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.793632    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istiod-ca-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.799909 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.794822    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-token" (OuterVolumeSpecName: "istio-token") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:02.801650 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.801615    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:44:02.801769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.801692    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-kube-api-access-rtff2" (OuterVolumeSpecName: "kube-api-access-rtff2") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "kube-api-access-rtff2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:02.801824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.801795    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "949ec5a0-f9ac-47a4-ab61-3f857c94ca41" (UID: "949ec5a0-f9ac-47a4-ab61-3f857c94ca41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:44:02.894797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.894747    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-envoy\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.894797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.894791    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-token\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.894797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.894806    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtff2\" (UniqueName: \"kubernetes.io/projected/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-kube-api-access-rtff2\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:02.895084 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:02.894822    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949ec5a0-f9ac-47a4-ab61-3f857c94ca41-istio-podinfo\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:03.121686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.121597    2562 generic.go:356] "Generic (PLEG): container finished" podID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerID="d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7" exitCode=0
Sep 10 19:44:03.121686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.121648    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" event={"ID":"949ec5a0-f9ac-47a4-ab61-3f857c94ca41","Type":"ContainerDied","Data":"d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7"}
Sep 10 19:44:03.121686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.121658    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"
Sep 10 19:44:03.121961 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.121723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg" event={"ID":"949ec5a0-f9ac-47a4-ab61-3f857c94ca41","Type":"ContainerDied","Data":"bde9691307c914edeb0b353c8e5b819ca4dd02746cf3fbc14942961a8d12da1e"}
Sep 10 19:44:03.121961 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.121747    2562 scope.go:117] "RemoveContainer" containerID="d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7"
Sep 10 19:44:03.131641 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.131612    2562 scope.go:117] "RemoveContainer" containerID="1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12"
Sep 10 19:44:03.140151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.140130    2562 scope.go:117] "RemoveContainer" containerID="d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7"
Sep 10 19:44:03.140485 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:03.140463    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7\": container with ID starting with d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7 not found: ID does not exist" containerID="d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7"
Sep 10 19:44:03.140535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.140494    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7"} err="failed to get container status \"d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7\": rpc error: code = NotFound desc = could not find container \"d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7\": container with ID starting with d13bdfe967e3b1a04662d9ad252333ca3b1cfa93b98d96e2972fcf247c29edd7 not found: ID does not exist"
Sep 10 19:44:03.140535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.140513    2562 scope.go:117] "RemoveContainer" containerID="1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12"
Sep 10 19:44:03.140767 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:03.140751    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12\": container with ID starting with 1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12 not found: ID does not exist" containerID="1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12"
Sep 10 19:44:03.140816 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.140771    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12"} err="failed to get container status \"1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12\": rpc error: code = NotFound desc = could not find container \"1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12\": container with ID starting with 1eb858f05e94fbf5bafcf962634d6ff2ffbbb8eb8f505091dc3ea13676175b12 not found: ID does not exist"
Sep 10 19:44:03.145929 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.145900    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"]
Sep 10 19:44:03.152411 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.152385    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b97a9c60-openshift-default-d86c4f664-frswg"]
Sep 10 19:44:03.696158 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:03.696124    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" path="/var/lib/kubelet/pods/949ec5a0-f9ac-47a4-ab61-3f857c94ca41/volumes"
Sep 10 19:44:07.203841 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:07.203806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:07.288773 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:07.288724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:07.888867 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:07.888820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:14.142356 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:14.142320    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:14.249773 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:14.249736    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:14.379543 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:14.379492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:19.808538 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:19.808504    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:44:24.901998 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:24.901958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:25.127794 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:25.127756    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:31.696382 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:31.696337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:32.701336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:32.701300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:34.999474 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:34.999431    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:44:46.082247 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:46.082207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:48.695186 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.695149    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7736-7a18f6"]
Sep 10 19:44:48.695570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.695554    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy"
Sep 10 19:44:48.695570 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.695568    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy"
Sep 10 19:44:48.695649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.695591    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy"
Sep 10 19:44:48.695649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.695596    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy"
Sep 10 19:44:48.695712 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.695650    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy"
Sep 10 19:44:48.695712 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.695661    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="949ec5a0-f9ac-47a4-ab61-3f857c94ca41" containerName="istio-proxy"
Sep 10 19:44:48.698765 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.698748    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:48.701824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.701796    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zp7h9\""
Sep 10 19:44:48.707192 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.707156    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7736-7a18f6"]
Sep 10 19:44:48.832818 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.832773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl6r7\" (UniqueName: \"kubernetes.io/projected/6f047d8e-66c4-4ba2-94f7-71227cf18ec1-kube-api-access-xl6r7\") pod \"test-curl-7736-7a18f6\" (UID: \"6f047d8e-66c4-4ba2-94f7-71227cf18ec1\") " pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:48.933612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.933567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl6r7\" (UniqueName: \"kubernetes.io/projected/6f047d8e-66c4-4ba2-94f7-71227cf18ec1-kube-api-access-xl6r7\") pod \"test-curl-7736-7a18f6\" (UID: \"6f047d8e-66c4-4ba2-94f7-71227cf18ec1\") " pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:48.944507 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:48.944474    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl6r7\" (UniqueName: \"kubernetes.io/projected/6f047d8e-66c4-4ba2-94f7-71227cf18ec1-kube-api-access-xl6r7\") pod \"test-curl-7736-7a18f6\" (UID: \"6f047d8e-66c4-4ba2-94f7-71227cf18ec1\") " pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:49.009585 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:49.009478    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:49.284483 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:49.284453    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7736-7a18f6"]
Sep 10 19:44:49.286720 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:44:49.286695    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f047d8e_66c4_4ba2_94f7_71227cf18ec1.slice/crio-24760ad36b8149956246dd4ef2cd3400237cf39fd2941454bcd934f5b21b2d77 WatchSource:0}: Error finding container 24760ad36b8149956246dd4ef2cd3400237cf39fd2941454bcd934f5b21b2d77: Status 404 returned error can't find the container with id 24760ad36b8149956246dd4ef2cd3400237cf39fd2941454bcd934f5b21b2d77
Sep 10 19:44:49.367629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:49.367595    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7736-7a18f6" event={"ID":"6f047d8e-66c4-4ba2-94f7-71227cf18ec1","Type":"ContainerStarted","Data":"9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2"}
Sep 10 19:44:49.367783 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:49.367642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7736-7a18f6" event={"ID":"6f047d8e-66c4-4ba2-94f7-71227cf18ec1","Type":"ContainerStarted","Data":"24760ad36b8149956246dd4ef2cd3400237cf39fd2941454bcd934f5b21b2d77"}
Sep 10 19:44:49.383938 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:49.383860    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7736-7a18f6" podStartSLOduration=1.3838413649999999 podStartE2EDuration="1.383841365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:44:49.383426316 +0000 UTC m=+1482.323460150" watchObservedRunningTime="2026-09-10 19:44:49.383841365 +0000 UTC m=+1482.323875199"
Sep 10 19:44:50.008246 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.008198    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7736-7a18f6"]
Sep 10 19:44:50.012715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.012682    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7736-7a18f6"]
Sep 10 19:44:50.054874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.054768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:50.127450 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.127408    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7736-f3d82d"]
Sep 10 19:44:50.127851 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.127837    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f047d8e-66c4-4ba2-94f7-71227cf18ec1" containerName="test-curl-7736-7a18f6"
Sep 10 19:44:50.127919 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.127852    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f047d8e-66c4-4ba2-94f7-71227cf18ec1" containerName="test-curl-7736-7a18f6"
Sep 10 19:44:50.127975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.127964    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f047d8e-66c4-4ba2-94f7-71227cf18ec1" containerName="test-curl-7736-7a18f6"
Sep 10 19:44:50.131093 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.131077    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:50.138517 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.138272    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7736-f3d82d"]
Sep 10 19:44:50.246331 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.246292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84xfs\" (UniqueName: \"kubernetes.io/projected/2f2bb6e0-8b91-4af8-9048-743a66f0fd29-kube-api-access-84xfs\") pod \"test-curl-7736-f3d82d\" (UID: \"2f2bb6e0-8b91-4af8-9048-743a66f0fd29\") " pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:50.347268 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.347179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84xfs\" (UniqueName: \"kubernetes.io/projected/2f2bb6e0-8b91-4af8-9048-743a66f0fd29-kube-api-access-84xfs\") pod \"test-curl-7736-f3d82d\" (UID: \"2f2bb6e0-8b91-4af8-9048-743a66f0fd29\") " pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:50.356941 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.356908    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84xfs\" (UniqueName: \"kubernetes.io/projected/2f2bb6e0-8b91-4af8-9048-743a66f0fd29-kube-api-access-84xfs\") pod \"test-curl-7736-f3d82d\" (UID: \"2f2bb6e0-8b91-4af8-9048-743a66f0fd29\") " pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:50.372475 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:50.372445    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7736-7a18f6" container="test-curl-7736-7a18f6"
Sep 10 19:44:50.441105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.441065    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:50.712213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:50.712187    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7736-f3d82d"]
Sep 10 19:44:50.714938 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:44:50.714878    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f2bb6e0_8b91_4af8_9048_743a66f0fd29.slice/crio-b96020ce0cca1b3528ed462519441ac5ab0b256c55b8f1bc83c879f6e75702c7 WatchSource:0}: Error finding container b96020ce0cca1b3528ed462519441ac5ab0b256c55b8f1bc83c879f6e75702c7: Status 404 returned error can't find the container with id b96020ce0cca1b3528ed462519441ac5ab0b256c55b8f1bc83c879f6e75702c7
Sep 10 19:44:51.380293 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:51.380256    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7736-f3d82d" event={"ID":"2f2bb6e0-8b91-4af8-9048-743a66f0fd29","Type":"ContainerStarted","Data":"5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae"}
Sep 10 19:44:51.380293 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:51.380298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7736-f3d82d" event={"ID":"2f2bb6e0-8b91-4af8-9048-743a66f0fd29","Type":"ContainerStarted","Data":"b96020ce0cca1b3528ed462519441ac5ab0b256c55b8f1bc83c879f6e75702c7"}
Sep 10 19:44:51.380841 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:51.380374    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7736-7a18f6" podUID="6f047d8e-66c4-4ba2-94f7-71227cf18ec1" containerName="test-curl-7736-7a18f6" containerID="cri-o://9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2" gracePeriod=2
Sep 10 19:44:51.395836 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:51.395778    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7736-f3d82d" podStartSLOduration=1.395760074 podStartE2EDuration="1.395760074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:44:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:44:51.394929249 +0000 UTC m=+1484.334963078" watchObservedRunningTime="2026-09-10 19:44:51.395760074 +0000 UTC m=+1484.335793906"
Sep 10 19:44:51.397420 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:51.397396    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7736-7a18f6\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="6f047d8e-66c4-4ba2-94f7-71227cf18ec1" pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:52.049135 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:52.049098    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7736-f3d82d"]
Sep 10 19:44:52.055139 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:52.055110    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7736-f3d82d"]
Sep 10 19:44:52.209522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:52.209480    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:52.309698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:52.309659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:52.385630 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:52.385597    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7736-f3d82d" container="test-curl-7736-f3d82d"
Sep 10 19:44:52.654575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:52.654537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:52.946218 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:52.946123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:53.164835 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:53.164797    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:53.391047 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:53.391012    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7736-f3d82d" podUID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" containerName="test-curl-7736-f3d82d" containerID="cri-o://5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae" gracePeriod=2
Sep 10 19:44:53.395237 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:53.395208    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7736-f3d82d\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:53.575205 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:53.575173    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:53.577598 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:53.577569    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7736-f3d82d\" is forbidden: User \"system:node:ip-10-0-139-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-35.ec2.internal' and this object" podUID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:53.676873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:53.676837    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl6r7\" (UniqueName: \"kubernetes.io/projected/6f047d8e-66c4-4ba2-94f7-71227cf18ec1-kube-api-access-xl6r7\") pod \"6f047d8e-66c4-4ba2-94f7-71227cf18ec1\" (UID: \"6f047d8e-66c4-4ba2-94f7-71227cf18ec1\") "
Sep 10 19:44:53.679058 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:53.679032    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f047d8e-66c4-4ba2-94f7-71227cf18ec1-kube-api-access-xl6r7" (OuterVolumeSpecName: "kube-api-access-xl6r7") pod "6f047d8e-66c4-4ba2-94f7-71227cf18ec1" (UID: "6f047d8e-66c4-4ba2-94f7-71227cf18ec1"). InnerVolumeSpecName "kube-api-access-xl6r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:53.695856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:53.695825    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f047d8e-66c4-4ba2-94f7-71227cf18ec1" path="/var/lib/kubelet/pods/6f047d8e-66c4-4ba2-94f7-71227cf18ec1/volumes"
Sep 10 19:44:53.778691 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:53.778644    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl6r7\" (UniqueName: \"kubernetes.io/projected/6f047d8e-66c4-4ba2-94f7-71227cf18ec1-kube-api-access-xl6r7\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:54.108724 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.108618    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:44:54.397937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.397896    2562 generic.go:356] "Generic (PLEG): container finished" podID="6f047d8e-66c4-4ba2-94f7-71227cf18ec1" containerID="9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2" exitCode=137
Sep 10 19:44:54.398414 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.397954    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-7a18f6"
Sep 10 19:44:54.398414 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.397964    2562 scope.go:117] "RemoveContainer" containerID="9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2"
Sep 10 19:44:54.406630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.406608    2562 scope.go:117] "RemoveContainer" containerID="9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2"
Sep 10 19:44:54.407009 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:54.406974    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2\": container with ID starting with 9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2 not found: ID does not exist" containerID="9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2"
Sep 10 19:44:54.407115 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.407014    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2"} err="failed to get container status \"9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2\": rpc error: code = NotFound desc = could not find container \"9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2\": container with ID starting with 9df6c3a996bb5130ebeee2426d88afea6f1750a721e82767f69e14929b5422c2 not found: ID does not exist"
Sep 10 19:44:54.670630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.670540    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"]
Sep 10 19:44:54.691052 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:54.691018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.564134 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:55.564106    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:55.662282 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:55.662193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.698353 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:55.698325    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84xfs\" (UniqueName: \"kubernetes.io/projected/2f2bb6e0-8b91-4af8-9048-743a66f0fd29-kube-api-access-84xfs\") pod \"2f2bb6e0-8b91-4af8-9048-743a66f0fd29\" (UID: \"2f2bb6e0-8b91-4af8-9048-743a66f0fd29\") "
Sep 10 19:44:55.700376 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:55.700349    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f2bb6e0-8b91-4af8-9048-743a66f0fd29-kube-api-access-84xfs" (OuterVolumeSpecName: "kube-api-access-84xfs") pod "2f2bb6e0-8b91-4af8-9048-743a66f0fd29" (UID: "2f2bb6e0-8b91-4af8-9048-743a66f0fd29"). InnerVolumeSpecName "kube-api-access-84xfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:55.752404 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:55.752366    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:55.799244 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:55.799204    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84xfs\" (UniqueName: \"kubernetes.io/projected/2f2bb6e0-8b91-4af8-9048-743a66f0fd29-kube-api-access-84xfs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:44:55.989611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:55.989525    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:56.412139 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:56.412102    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" containerID="5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae" exitCode=137
Sep 10 19:44:56.412356 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:56.412147    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7736-f3d82d"
Sep 10 19:44:56.412356 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:56.412202    2562 scope.go:117] "RemoveContainer" containerID="5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae"
Sep 10 19:44:56.421153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:56.421135    2562 scope.go:117] "RemoveContainer" containerID="5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae"
Sep 10 19:44:56.421439 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:44:56.421418    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae\": container with ID starting with 5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae not found: ID does not exist" containerID="5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae"
Sep 10 19:44:56.421486 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:56.421449    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae"} err="failed to get container status \"5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae\": rpc error: code = NotFound desc = could not find container \"5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae\": container with ID starting with 5371d48a5095c96c494279704deac9441104f661db4876641b6075d336fe74ae not found: ID does not exist"
Sep 10 19:44:57.059453 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.059407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:57.159814 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.159781    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:57.282539 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.282500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:57.624803 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.624768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:57.696936 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.696880    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" path="/var/lib/kubelet/pods/2f2bb6e0-8b91-4af8-9048-743a66f0fd29/volumes"
Sep 10 19:44:57.719443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.719413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:57.762340 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.762303    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"]
Sep 10 19:44:57.762744 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.762730    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" containerName="test-curl-7736-f3d82d"
Sep 10 19:44:57.762789 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.762746    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" containerName="test-curl-7736-f3d82d"
Sep 10 19:44:57.762839 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.762829    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2bb6e0-8b91-4af8-9048-743a66f0fd29" containerName="test-curl-7736-f3d82d"
Sep 10 19:44:57.767240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.767220    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:57.769953 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.769927    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d77d5a\""
Sep 10 19:44:57.770350 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.770333    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d77d5a-dockercfg-crwph\""
Sep 10 19:44:57.779210 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.779185    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"]
Sep 10 19:44:57.810615 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.810576    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"]
Sep 10 19:44:57.814059 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.814042    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:57.816612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.816587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:57.816733 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.816628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6w5m\" (UniqueName: \"kubernetes.io/projected/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-kube-api-access-f6w5m\") pod \"payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:57.827039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.827007    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"]
Sep 10 19:44:57.917920 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.917844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d554a0a-5608-4a46-ae75-6502f7bb99af-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") " pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:57.918100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.917933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89hgz\" (UniqueName: \"kubernetes.io/projected/7d554a0a-5608-4a46-ae75-6502f7bb99af-kube-api-access-89hgz\") pod \"payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") " pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:57.918100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.917960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6w5m\" (UniqueName: \"kubernetes.io/projected/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-kube-api-access-f6w5m\") pod \"payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:57.918187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.918101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:57.918807 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.918785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:57.926877 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:57.926850    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6w5m\" (UniqueName: \"kubernetes.io/projected/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-kube-api-access-f6w5m\") pod \"payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:58.019303 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.019258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d554a0a-5608-4a46-ae75-6502f7bb99af-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") " pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:58.019512 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.019347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89hgz\" (UniqueName: \"kubernetes.io/projected/7d554a0a-5608-4a46-ae75-6502f7bb99af-kube-api-access-89hgz\") pod \"payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") " pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:58.020116 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.020093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d554a0a-5608-4a46-ae75-6502f7bb99af-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") " pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:58.028455 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.028429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89hgz\" (UniqueName: \"kubernetes.io/projected/7d554a0a-5608-4a46-ae75-6502f7bb99af-kube-api-access-89hgz\") pod \"payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") " pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:58.078078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.078038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:58.122287 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.122246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:44:58.123822 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.123799    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:58.361203 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.361171    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"]
Sep 10 19:44:58.363155 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:44:58.363121    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d554a0a_5608_4a46_ae75_6502f7bb99af.slice/crio-7d35fc1abc88244300c2f22af4c1549925bb9f026b4340bee4223e3254efbe94 WatchSource:0}: Error finding container 7d35fc1abc88244300c2f22af4c1549925bb9f026b4340bee4223e3254efbe94: Status 404 returned error can't find the container with id 7d35fc1abc88244300c2f22af4c1549925bb9f026b4340bee4223e3254efbe94
Sep 10 19:44:58.416856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.416827    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"]
Sep 10 19:44:58.417117 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:44:58.417096    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ff7df9_ab24_46c2_88f1_1d0590f136b6.slice/crio-cb70346cdbadd187997ac5afe1e5adbe18e4a2aee878dddd72e85c614fbfd83b WatchSource:0}: Error finding container cb70346cdbadd187997ac5afe1e5adbe18e4a2aee878dddd72e85c614fbfd83b: Status 404 returned error can't find the container with id cb70346cdbadd187997ac5afe1e5adbe18e4a2aee878dddd72e85c614fbfd83b
Sep 10 19:44:58.423854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.423826    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62" event={"ID":"f4ff7df9-ab24-46c2-88f1-1d0590f136b6","Type":"ContainerStarted","Data":"cb70346cdbadd187997ac5afe1e5adbe18e4a2aee878dddd72e85c614fbfd83b"}
Sep 10 19:44:58.425580 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:58.425550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd" event={"ID":"7d554a0a-5608-4a46-ae75-6502f7bb99af","Type":"ContainerStarted","Data":"7d35fc1abc88244300c2f22af4c1549925bb9f026b4340bee4223e3254efbe94"}
Sep 10 19:44:59.432779 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:59.432732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62" event={"ID":"f4ff7df9-ab24-46c2-88f1-1d0590f136b6","Type":"ContainerStarted","Data":"812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51"}
Sep 10 19:44:59.433277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:59.432813    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:44:59.434511 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:59.434482    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd" event={"ID":"7d554a0a-5608-4a46-ae75-6502f7bb99af","Type":"ContainerStarted","Data":"58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d"}
Sep 10 19:44:59.434614 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:59.434540    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:44:59.452286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:59.452225    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62" podStartSLOduration=2.452209021 podStartE2EDuration="2.452209021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:44:59.451011915 +0000 UTC m=+1492.391045765" watchObservedRunningTime="2026-09-10 19:44:59.452209021 +0000 UTC m=+1492.392242855"
Sep 10 19:44:59.474796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:59.474738    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd" podStartSLOduration=2.474722499 podStartE2EDuration="2.474722499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:44:59.474613607 +0000 UTC m=+1492.414647441" watchObservedRunningTime="2026-09-10 19:44:59.474722499 +0000 UTC m=+1492.414756331"
Sep 10 19:44:59.858364 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:44:59.858248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:00.153065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.153024    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"]
Sep 10 19:45:00.157412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.157389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"
Sep 10 19:45:00.174616 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.174580    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:45:00.177407 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.177376    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"]
Sep 10 19:45:00.178300 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.178274    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"
Sep 10 19:45:00.180138 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.180112    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"]
Sep 10 19:45:00.184343 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.184321    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"
Sep 10 19:45:00.193013 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.192977    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"]
Sep 10 19:45:00.197117 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.197095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"
Sep 10 19:45:00.210752 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.210655    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"]
Sep 10 19:45:00.221214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.221175    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"]
Sep 10 19:45:00.223171 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.223141    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"
Sep 10 19:45:00.232871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.232842    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"
Sep 10 19:45:00.468153 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.468124    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"]
Sep 10 19:45:00.469570 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:45:00.469548    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e7cdae_89e1_4e73_bfcf_bb7fec6c061f.slice/crio-5c6a16ed4e819d8996c8e6be122b5384df4b8fb6fc3aa14c0093049b53af3869 WatchSource:0}: Error finding container 5c6a16ed4e819d8996c8e6be122b5384df4b8fb6fc3aa14c0093049b53af3869: Status 404 returned error can't find the container with id 5c6a16ed4e819d8996c8e6be122b5384df4b8fb6fc3aa14c0093049b53af3869
Sep 10 19:45:00.527355 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.527321    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"]
Sep 10 19:45:00.528999 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:45:00.528966    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1d93e0_a744_473c_88e7_f7bab356755e.slice/crio-3662977f114efc59980e61a3da8cef576c6dbe986c6e43e7f3d228076eab42b7 WatchSource:0}: Error finding container 3662977f114efc59980e61a3da8cef576c6dbe986c6e43e7f3d228076eab42b7: Status 404 returned error can't find the container with id 3662977f114efc59980e61a3da8cef576c6dbe986c6e43e7f3d228076eab42b7
Sep 10 19:45:00.529760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:00.529691    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"]
Sep 10 19:45:00.530026 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:45:00.530004    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce7493a_4880_48da_810e_5220ad4c5af8.slice/crio-2ef385666f88619fea5838e35f3f7fb2f3c59f444fb4a04893f005df1d8f8fa3 WatchSource:0}: Error finding container 2ef385666f88619fea5838e35f3f7fb2f3c59f444fb4a04893f005df1d8f8fa3: Status 404 returned error can't find the container with id 2ef385666f88619fea5838e35f3f7fb2f3c59f444fb4a04893f005df1d8f8fa3
Sep 10 19:45:01.449003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.448853    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6" event={"ID":"7ce7493a-4880-48da-810e-5220ad4c5af8","Type":"ContainerStarted","Data":"fed444edee309ca6680acb2678a2618a11be7aabb68689bc391952b3a9c3e01f"}
Sep 10 19:45:01.449003 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.448981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6" event={"ID":"7ce7493a-4880-48da-810e-5220ad4c5af8","Type":"ContainerStarted","Data":"2ef385666f88619fea5838e35f3f7fb2f3c59f444fb4a04893f005df1d8f8fa3"}
Sep 10 19:45:01.450591 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.450558    2562 generic.go:356] "Generic (PLEG): container finished" podID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerID="dfa68f450b23c60ced7b6d224e0dfc04ea41880a4c276a44da2ad59c901063c1" exitCode=7
Sep 10 19:45:01.450729 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.450644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv" event={"ID":"a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f","Type":"ContainerDied","Data":"dfa68f450b23c60ced7b6d224e0dfc04ea41880a4c276a44da2ad59c901063c1"}
Sep 10 19:45:01.450729 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.450699    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv" event={"ID":"a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f","Type":"ContainerStarted","Data":"5c6a16ed4e819d8996c8e6be122b5384df4b8fb6fc3aa14c0093049b53af3869"}
Sep 10 19:45:01.450835 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.450783    2562 scope.go:117] "RemoveContainer" containerID="dfa68f450b23c60ced7b6d224e0dfc04ea41880a4c276a44da2ad59c901063c1"
Sep 10 19:45:01.452496 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.452471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6" event={"ID":"fc1d93e0-a744-473c-88e7-f7bab356755e","Type":"ContainerStarted","Data":"47af9f460a78160c999fd7c020707ec5586ce5609fb2dd565d5781d5b70c7c11"}
Sep 10 19:45:01.452611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.452519    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6" event={"ID":"fc1d93e0-a744-473c-88e7-f7bab356755e","Type":"ContainerStarted","Data":"3662977f114efc59980e61a3da8cef576c6dbe986c6e43e7f3d228076eab42b7"}
Sep 10 19:45:01.466182 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:01.466131    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6" podStartSLOduration=1.466113819 podStartE2EDuration="1.466113819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:45:01.464649471 +0000 UTC m=+1494.404683305" watchObservedRunningTime="2026-09-10 19:45:01.466113819 +0000 UTC m=+1494.406147652"
Sep 10 19:45:02.460684 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.460592    2562 generic.go:356] "Generic (PLEG): container finished" podID="fc1d93e0-a744-473c-88e7-f7bab356755e" containerID="47af9f460a78160c999fd7c020707ec5586ce5609fb2dd565d5781d5b70c7c11" exitCode=0
Sep 10 19:45:02.460684 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.460670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6" event={"ID":"fc1d93e0-a744-473c-88e7-f7bab356755e","Type":"ContainerDied","Data":"47af9f460a78160c999fd7c020707ec5586ce5609fb2dd565d5781d5b70c7c11"}
Sep 10 19:45:02.462428 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.462402    2562 generic.go:356] "Generic (PLEG): container finished" podID="7ce7493a-4880-48da-810e-5220ad4c5af8" containerID="fed444edee309ca6680acb2678a2618a11be7aabb68689bc391952b3a9c3e01f" exitCode=0
Sep 10 19:45:02.462507 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.462481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6" event={"ID":"7ce7493a-4880-48da-810e-5220ad4c5af8","Type":"ContainerDied","Data":"fed444edee309ca6680acb2678a2618a11be7aabb68689bc391952b3a9c3e01f"}
Sep 10 19:45:02.464487 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.464462    2562 generic.go:356] "Generic (PLEG): container finished" podID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerID="a1435210229c076821afa7d74cf953b248ae2c69a0124cabfa21e76e9a1ddd4b" exitCode=7
Sep 10 19:45:02.464612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.464499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv" event={"ID":"a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f","Type":"ContainerDied","Data":"a1435210229c076821afa7d74cf953b248ae2c69a0124cabfa21e76e9a1ddd4b"}
Sep 10 19:45:02.464612 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.464535    2562 scope.go:117] "RemoveContainer" containerID="dfa68f450b23c60ced7b6d224e0dfc04ea41880a4c276a44da2ad59c901063c1"
Sep 10 19:45:02.464699 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:02.464685    2562 scope.go:117] "RemoveContainer" containerID="a1435210229c076821afa7d74cf953b248ae2c69a0124cabfa21e76e9a1ddd4b"
Sep 10 19:45:02.464987 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:45:02.464968    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv_odh-ai-gateway-infra(a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f"
Sep 10 19:45:03.472214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:03.472181    2562 scope.go:117] "RemoveContainer" containerID="a1435210229c076821afa7d74cf953b248ae2c69a0124cabfa21e76e9a1ddd4b"
Sep 10 19:45:03.472744 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:45:03.472435    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv_odh-ai-gateway-infra(a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f"
Sep 10 19:45:03.619688 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:03.619658    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"
Sep 10 19:45:03.654909 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:03.654868    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"
Sep 10 19:45:04.282620 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.282578    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"]
Sep 10 19:45:04.283237 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.283208    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc1d93e0-a744-473c-88e7-f7bab356755e" containerName="cleanup"
Sep 10 19:45:04.283237 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.283235    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc1d93e0-a744-473c-88e7-f7bab356755e" containerName="cleanup"
Sep 10 19:45:04.283427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.283252    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce7493a-4880-48da-810e-5220ad4c5af8" containerName="cleanup"
Sep 10 19:45:04.283427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.283259    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce7493a-4880-48da-810e-5220ad4c5af8" containerName="cleanup"
Sep 10 19:45:04.283427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.283320    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce7493a-4880-48da-810e-5220ad4c5af8" containerName="cleanup"
Sep 10 19:45:04.283427 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.283333    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc1d93e0-a744-473c-88e7-f7bab356755e" containerName="cleanup"
Sep 10 19:45:04.286521 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.286498    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.289360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.289334    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-07c5fe\"/\"rate-test-model-07c5fe-kserve-self-signed-certs\""
Sep 10 19:45:04.296687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.296660    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"]
Sep 10 19:45:04.315200 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.315150    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:04.378360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.378323    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-model-cache\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.378547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.378382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-home\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.378547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.378409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-dshm\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.378547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.378503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b8658c-6703-495f-a213-f1c600d37351-tls-certs\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.378652 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.378546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp6s5\" (UniqueName: \"kubernetes.io/projected/98b8658c-6703-495f-a213-f1c600d37351-kube-api-access-dp6s5\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.378652 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.378567    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-tmp-dir\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.478959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.478924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6" event={"ID":"fc1d93e0-a744-473c-88e7-f7bab356755e","Type":"ContainerDied","Data":"3662977f114efc59980e61a3da8cef576c6dbe986c6e43e7f3d228076eab42b7"}
Sep 10 19:45:04.478959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.478954    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"
Sep 10 19:45:04.479458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.478978    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3662977f114efc59980e61a3da8cef576c6dbe986c6e43e7f3d228076eab42b7"
Sep 10 19:45:04.479458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b8658c-6703-495f-a213-f1c600d37351-tls-certs\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp6s5\" (UniqueName: \"kubernetes.io/projected/98b8658c-6703-495f-a213-f1c600d37351-kube-api-access-dp6s5\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-tmp-dir\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-model-cache\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-home\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479240    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-dshm\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-model-cache\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479824 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479750    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-home\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.479940 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.479822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-tmp-dir\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.480983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.480874    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6" event={"ID":"7ce7493a-4880-48da-810e-5220ad4c5af8","Type":"ContainerDied","Data":"2ef385666f88619fea5838e35f3f7fb2f3c59f444fb4a04893f005df1d8f8fa3"}
Sep 10 19:45:04.480983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.480926    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"
Sep 10 19:45:04.480983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.480947    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ef385666f88619fea5838e35f3f7fb2f3c59f444fb4a04893f005df1d8f8fa3"
Sep 10 19:45:04.481766 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.481748    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-dshm\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.481967 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.481945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b8658c-6703-495f-a213-f1c600d37351-tls-certs\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.488258 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.488233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp6s5\" (UniqueName: \"kubernetes.io/projected/98b8658c-6703-495f-a213-f1c600d37351-kube-api-access-dp6s5\") pod \"rate-test-model-07c5fe-kserve-55bb6df899-8hfvx\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") " pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.597127 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.597030    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:04.873169 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:04.873138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"]
Sep 10 19:45:04.875076 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:45:04.875046    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98b8658c_6703_495f_a213_f1c600d37351.slice/crio-831f86d2be706a90f9d4d46b5b9d48299ad020115778589b5f0dcc5448d0af8f WatchSource:0}: Error finding container 831f86d2be706a90f9d4d46b5b9d48299ad020115778589b5f0dcc5448d0af8f: Status 404 returned error can't find the container with id 831f86d2be706a90f9d4d46b5b9d48299ad020115778589b5f0dcc5448d0af8f
Sep 10 19:45:05.487852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:05.487817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx" event={"ID":"98b8658c-6703-495f-a213-f1c600d37351","Type":"ContainerStarted","Data":"2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa"}
Sep 10 19:45:05.487852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:05.487862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx" event={"ID":"98b8658c-6703-495f-a213-f1c600d37351","Type":"ContainerStarted","Data":"831f86d2be706a90f9d4d46b5b9d48299ad020115778589b5f0dcc5448d0af8f"}
Sep 10 19:45:05.508462 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:05.508399    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx" podStartSLOduration=1.508379536 podStartE2EDuration="1.508379536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:45:05.505656467 +0000 UTC m=+1498.445690299" watchObservedRunningTime="2026-09-10 19:45:05.508379536 +0000 UTC m=+1498.448413368"
Sep 10 19:45:07.656771 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:07.656733    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:45:07.661666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:07.661643    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:45:07.812989 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:07.812960    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:45:07.818641 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:07.818619    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:45:08.002835 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:08.002734    2562 scope.go:117] "RemoveContainer" containerID="6a6a2ad4f774b9a17e9883a20d3fd28aff0236dc981f06f37714408723246e45"
Sep 10 19:45:08.011683 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:08.011661    2562 scope.go:117] "RemoveContainer" containerID="8edcea87d63e0ffd65a426cd4354bbde38e9593e579d806a5e73e8432227616c"
Sep 10 19:45:09.690607 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:09.690564    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc" podUID="9e88bf98-08fb-4af9-b741-d129d2aa0006" containerName="main" containerID="cri-o://5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac" gracePeriod=45
Sep 10 19:45:09.938266 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:09.938238    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:45:10.038274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038186    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-dshm\") pod \"9e88bf98-08fb-4af9-b741-d129d2aa0006\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") "
Sep 10 19:45:10.038274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038232    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e88bf98-08fb-4af9-b741-d129d2aa0006-tls-certs\") pod \"9e88bf98-08fb-4af9-b741-d129d2aa0006\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") "
Sep 10 19:45:10.038274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038265    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9w86\" (UniqueName: \"kubernetes.io/projected/9e88bf98-08fb-4af9-b741-d129d2aa0006-kube-api-access-b9w86\") pod \"9e88bf98-08fb-4af9-b741-d129d2aa0006\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") "
Sep 10 19:45:10.038677 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038343    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-home\") pod \"9e88bf98-08fb-4af9-b741-d129d2aa0006\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") "
Sep 10 19:45:10.038677 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038375    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-model-cache\") pod \"9e88bf98-08fb-4af9-b741-d129d2aa0006\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") "
Sep 10 19:45:10.038677 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038402    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-tmp-dir\") pod \"9e88bf98-08fb-4af9-b741-d129d2aa0006\" (UID: \"9e88bf98-08fb-4af9-b741-d129d2aa0006\") "
Sep 10 19:45:10.038921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038673    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-home" (OuterVolumeSpecName: "home") pod "9e88bf98-08fb-4af9-b741-d129d2aa0006" (UID: "9e88bf98-08fb-4af9-b741-d129d2aa0006"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:10.038921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038688    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-model-cache" (OuterVolumeSpecName: "model-cache") pod "9e88bf98-08fb-4af9-b741-d129d2aa0006" (UID: "9e88bf98-08fb-4af9-b741-d129d2aa0006"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:10.038921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038776    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9e88bf98-08fb-4af9-b741-d129d2aa0006" (UID: "9e88bf98-08fb-4af9-b741-d129d2aa0006"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:10.038921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038836    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:45:10.038921 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.038856    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:45:10.040548 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.040516    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e88bf98-08fb-4af9-b741-d129d2aa0006-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9e88bf98-08fb-4af9-b741-d129d2aa0006" (UID: "9e88bf98-08fb-4af9-b741-d129d2aa0006"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:45:10.040660 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.040558    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-dshm" (OuterVolumeSpecName: "dshm") pod "9e88bf98-08fb-4af9-b741-d129d2aa0006" (UID: "9e88bf98-08fb-4af9-b741-d129d2aa0006"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:10.040660 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.040582    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e88bf98-08fb-4af9-b741-d129d2aa0006-kube-api-access-b9w86" (OuterVolumeSpecName: "kube-api-access-b9w86") pod "9e88bf98-08fb-4af9-b741-d129d2aa0006" (UID: "9e88bf98-08fb-4af9-b741-d129d2aa0006"). InnerVolumeSpecName "kube-api-access-b9w86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:10.139695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.139653    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:45:10.139695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.139691    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e88bf98-08fb-4af9-b741-d129d2aa0006-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:45:10.139695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.139700    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e88bf98-08fb-4af9-b741-d129d2aa0006-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:45:10.139695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.139710    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9w86\" (UniqueName: \"kubernetes.io/projected/9e88bf98-08fb-4af9-b741-d129d2aa0006-kube-api-access-b9w86\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:45:10.442139 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.442102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:45:10.442444 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.442419    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:45:10.516625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.516590    2562 generic.go:356] "Generic (PLEG): container finished" podID="9e88bf98-08fb-4af9-b741-d129d2aa0006" containerID="5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac" exitCode=0
Sep 10 19:45:10.516857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.516645    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"
Sep 10 19:45:10.516857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.516672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc" event={"ID":"9e88bf98-08fb-4af9-b741-d129d2aa0006","Type":"ContainerDied","Data":"5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac"}
Sep 10 19:45:10.516857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.516723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc" event={"ID":"9e88bf98-08fb-4af9-b741-d129d2aa0006","Type":"ContainerDied","Data":"41518f168a1d865fc45140c739233a487a6664a6087492023c98ee299ef5bf45"}
Sep 10 19:45:10.516857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.516753    2562 scope.go:117] "RemoveContainer" containerID="5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac"
Sep 10 19:45:10.526869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.526846    2562 scope.go:117] "RemoveContainer" containerID="5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac"
Sep 10 19:45:10.527221 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:45:10.527202    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac\": container with ID starting with 5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac not found: ID does not exist" containerID="5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac"
Sep 10 19:45:10.527277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.527231    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac"} err="failed to get container status \"5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac\": rpc error: code = NotFound desc = could not find container \"5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac\": container with ID starting with 5ee1cd8ad267cb940b8ec00e8d7c4aa92375796e002f04370e83a3c46acd76ac not found: ID does not exist"
Sep 10 19:45:10.539419 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.539384    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"]
Sep 10 19:45:10.545508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:10.545478    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/sub-test-model-07c5fe-kserve-595b79ffbc-fvllc"]
Sep 10 19:45:11.696021 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:11.695972    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e88bf98-08fb-4af9-b741-d129d2aa0006" path="/var/lib/kubelet/pods/9e88bf98-08fb-4af9-b741-d129d2aa0006/volumes"
Sep 10 19:45:11.912942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:11.912880    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:45:14.597944 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:14.597868    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:14.598362 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:14.597965    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:14.611308 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:14.611277    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:15.557526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:15.557492    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:45:18.689962 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:18.689922    2562 scope.go:117] "RemoveContainer" containerID="a1435210229c076821afa7d74cf953b248ae2c69a0124cabfa21e76e9a1ddd4b"
Sep 10 19:45:19.362914 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:19.362790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:19.571378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:19.571345    2562 generic.go:356] "Generic (PLEG): container finished" podID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerID="ff1b75852d75749360a86ee9dd64e7ae556b736998725ed84e79bafc531c0bd3" exitCode=0
Sep 10 19:45:19.571540 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:19.571404    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv" event={"ID":"a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f","Type":"ContainerDied","Data":"ff1b75852d75749360a86ee9dd64e7ae556b736998725ed84e79bafc531c0bd3"}
Sep 10 19:45:19.571540 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:19.571466    2562 scope.go:117] "RemoveContainer" containerID="a1435210229c076821afa7d74cf953b248ae2c69a0124cabfa21e76e9a1ddd4b"
Sep 10 19:45:20.603369 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:20.603329    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"]
Sep 10 19:45:20.627096 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:20.627057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:20.716644 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:20.716620    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"
Sep 10 19:45:21.108671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.108628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:21.435031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.434966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:21.585597 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.585568    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"
Sep 10 19:45:21.585772 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.585609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv" event={"ID":"a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f","Type":"ContainerDied","Data":"5c6a16ed4e819d8996c8e6be122b5384df4b8fb6fc3aa14c0093049b53af3869"}
Sep 10 19:45:21.585772 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.585661    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c6a16ed4e819d8996c8e6be122b5384df4b8fb6fc3aa14c0093049b53af3869"
Sep 10 19:45:21.624692 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.624648    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"]
Sep 10 19:45:21.630776 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.630743    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d77d5a-29817825-m4rmv"]
Sep 10 19:45:21.696094 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.695996    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" path="/var/lib/kubelet/pods/a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f/volumes"
Sep 10 19:45:21.762852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:21.762793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:23.889836 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:23.889786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:24.781625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:24.781575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:24.877500 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:24.877459    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:25.629337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:25.629297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:26.047725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.047685    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:26.147870 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.147816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:26.609797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.609760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:26.708712 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.708672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:26.775996 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.775954    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"]
Sep 10 19:45:26.776368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776355    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:26.776416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776369    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:26.776416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776378    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:26.776416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776384    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:26.776416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776394    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e88bf98-08fb-4af9-b741-d129d2aa0006" containerName="main"
Sep 10 19:45:26.776416 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776401    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e88bf98-08fb-4af9-b741-d129d2aa0006" containerName="main"
Sep 10 19:45:26.776616 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776472    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:26.776616 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776479    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:26.776616 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.776486    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e88bf98-08fb-4af9-b741-d129d2aa0006" containerName="main"
Sep 10 19:45:26.779781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.779763    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:26.782874 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.782849    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-261db5-dockercfg-dkjpl\""
Sep 10 19:45:26.783139 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.783123    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-261db5\""
Sep 10 19:45:26.789440 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.789412    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"]
Sep 10 19:45:26.810564 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.810524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:26.897995 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.897955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:26.898195 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.898028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4hqd\" (UniqueName: \"kubernetes.io/projected/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-kube-api-access-r4hqd\") pod \"payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:26.999143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.999102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:26.999332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.999157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4hqd\" (UniqueName: \"kubernetes.io/projected/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-kube-api-access-r4hqd\") pod \"payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:26.999832 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:26.999813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:27.012615 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:27.012589    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4hqd\" (UniqueName: \"kubernetes.io/projected/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-kube-api-access-r4hqd\") pod \"payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:27.091656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:27.091615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:27.369079 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:27.368924    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"]
Sep 10 19:45:27.376667 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:45:27.376638    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a26c900_3b04_40d1_a6fc_c1df4a12eea9.slice/crio-ad027e03d0c127577095206b3e3db13907bb9800ed2057c55d61b29b9a607942 WatchSource:0}: Error finding container ad027e03d0c127577095206b3e3db13907bb9800ed2057c55d61b29b9a607942: Status 404 returned error can't find the container with id ad027e03d0c127577095206b3e3db13907bb9800ed2057c55d61b29b9a607942
Sep 10 19:45:27.620255 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:27.620164    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs" event={"ID":"7a26c900-3b04-40d1-a6fc-c1df4a12eea9","Type":"ContainerStarted","Data":"5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01"}
Sep 10 19:45:27.620255 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:27.620220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs" event={"ID":"7a26c900-3b04-40d1-a6fc-c1df4a12eea9","Type":"ContainerStarted","Data":"ad027e03d0c127577095206b3e3db13907bb9800ed2057c55d61b29b9a607942"}
Sep 10 19:45:27.620448 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:27.620349    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:27.641257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:27.641198    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs" podStartSLOduration=1.6411809050000001 podStartE2EDuration="1.641180905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:45:27.638812898 +0000 UTC m=+1520.578846733" watchObservedRunningTime="2026-09-10 19:45:27.641180905 +0000 UTC m=+1520.581214737"
Sep 10 19:45:32.271028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.270950    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"]
Sep 10 19:45:32.271395 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.271382    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:32.271395 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.271395    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:32.271472 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.271463    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e7cdae-89e1-4e73-bfcf-bb7fec6c061f" containerName="cleanup"
Sep 10 19:45:32.274878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.274860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.280527 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.280495    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-2a1fa9\"/\"openshift-service-ca.crt\""
Sep 10 19:45:32.280669 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.280553    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-2a1fa9\"/\"default-dockercfg-ghwlj\""
Sep 10 19:45:32.281083 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.281057    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-2a1fa9\"/\"kube-root-ca.crt\""
Sep 10 19:45:32.281217 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.281180    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-2a1fa9\"/\"rate-test-model-2a1fa9-kserve-self-signed-certs\""
Sep 10 19:45:32.286984 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.286945    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"]
Sep 10 19:45:32.318150 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.318111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:32.452679 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.452638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-tmp-dir\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.452924 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.452701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlwvw\" (UniqueName: \"kubernetes.io/projected/439e5132-a506-4fb5-9479-5ad25e76d5ed-kube-api-access-nlwvw\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.452924 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.452743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-model-cache\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.452924 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.452766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-home\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.452924 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.452857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-dshm\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.452924 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.452883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/439e5132-a506-4fb5-9479-5ad25e76d5ed-tls-certs\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554062 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.553961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-dshm\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554062 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/439e5132-a506-4fb5-9479-5ad25e76d5ed-tls-certs\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-tmp-dir\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlwvw\" (UniqueName: \"kubernetes.io/projected/439e5132-a506-4fb5-9479-5ad25e76d5ed-kube-api-access-nlwvw\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-model-cache\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554170    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-home\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554545 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554519    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-tmp-dir\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554616 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554526    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-home\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.554666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.554604    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-model-cache\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.556463 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.556441    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-dshm\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.556606 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.556586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/439e5132-a506-4fb5-9479-5ad25e76d5ed-tls-certs\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.567269 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.567240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlwvw\" (UniqueName: \"kubernetes.io/projected/439e5132-a506-4fb5-9479-5ad25e76d5ed-kube-api-access-nlwvw\") pod \"rate-test-model-2a1fa9-kserve-f74687999-vh4nt\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") " pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.584125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.584085    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:32.865371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:32.865331    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"]
Sep 10 19:45:32.866524 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:45:32.866492    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod439e5132_a506_4fb5_9479_5ad25e76d5ed.slice/crio-37169f39c9b37277807dda5653ae92d0d46099498f43394f50ef60adb70333ce WatchSource:0}: Error finding container 37169f39c9b37277807dda5653ae92d0d46099498f43394f50ef60adb70333ce: Status 404 returned error can't find the container with id 37169f39c9b37277807dda5653ae92d0d46099498f43394f50ef60adb70333ce
Sep 10 19:45:33.656345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:33.656300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt" event={"ID":"439e5132-a506-4fb5-9479-5ad25e76d5ed","Type":"ContainerStarted","Data":"7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684"}
Sep 10 19:45:33.656345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:33.656353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt" event={"ID":"439e5132-a506-4fb5-9479-5ad25e76d5ed","Type":"ContainerStarted","Data":"37169f39c9b37277807dda5653ae92d0d46099498f43394f50ef60adb70333ce"}
Sep 10 19:45:33.676860 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:33.676800    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt" podStartSLOduration=1.67678378 podStartE2EDuration="1.67678378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:45:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:45:33.674983345 +0000 UTC m=+1526.615017173" watchObservedRunningTime="2026-09-10 19:45:33.67678378 +0000 UTC m=+1526.616817606"
Sep 10 19:45:36.501648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:36.501608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:45:38.628100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:38.628064    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:45:42.585195 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:42.585152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:42.585195 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:42.585207    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:42.598363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:42.598333    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:42.715822 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:42.715789    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:45:49.826628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:49.826584    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:51.837163 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:51.837124    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:53.843464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:53.843404    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"]
Sep 10 19:45:53.847322 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:53.847303    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"
Sep 10 19:45:53.850479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:53.850452    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:45:53.850601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:53.850544    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:45:53.855778 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:53.855538    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"]
Sep 10 19:45:53.942552 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:53.942512    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866fb2ea-76c5-48a7-88b4-502beeca87ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s\" (UID: \"866fb2ea-76c5-48a7-88b4-502beeca87ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"
Sep 10 19:45:53.957363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:53.957326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:54.043988 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.043957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866fb2ea-76c5-48a7-88b4-502beeca87ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s\" (UID: \"866fb2ea-76c5-48a7-88b4-502beeca87ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"
Sep 10 19:45:54.044581 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.044559    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866fb2ea-76c5-48a7-88b4-502beeca87ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s\" (UID: \"866fb2ea-76c5-48a7-88b4-502beeca87ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"
Sep 10 19:45:54.159810 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.159779    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"
Sep 10 19:45:54.435845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.435814    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"]
Sep 10 19:45:54.438076 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:45:54.438048    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866fb2ea_76c5_48a7_88b4_502beeca87ca.slice/crio-6c59ef421f5e21f876386688c66c6546609ea41171b20feaeec7f3bfdc3b9a87 WatchSource:0}: Error finding container 6c59ef421f5e21f876386688c66c6546609ea41171b20feaeec7f3bfdc3b9a87: Status 404 returned error can't find the container with id 6c59ef421f5e21f876386688c66c6546609ea41171b20feaeec7f3bfdc3b9a87
Sep 10 19:45:54.707078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.706985    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:45:54.770065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.770027    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s" event={"ID":"866fb2ea-76c5-48a7-88b4-502beeca87ca","Type":"ContainerStarted","Data":"06f81950e5bf0bd32ddb7f02e9ec603185641cbc7115e44315133af98e37e745"}
Sep 10 19:45:54.770248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.770090    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s" event={"ID":"866fb2ea-76c5-48a7-88b4-502beeca87ca","Type":"ContainerStarted","Data":"6c59ef421f5e21f876386688c66c6546609ea41171b20feaeec7f3bfdc3b9a87"}
Sep 10 19:45:54.787355 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:54.787295    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s" podStartSLOduration=1.78728061 podStartE2EDuration="1.78728061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:45:54.785878585 +0000 UTC m=+1547.725912410" watchObservedRunningTime="2026-09-10 19:45:54.78728061 +0000 UTC m=+1547.727314443"
Sep 10 19:45:55.095210 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:55.095175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:55.166941 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:55.166880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:45:55.776976 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:55.776938    2562 generic.go:356] "Generic (PLEG): container finished" podID="866fb2ea-76c5-48a7-88b4-502beeca87ca" containerID="06f81950e5bf0bd32ddb7f02e9ec603185641cbc7115e44315133af98e37e745" exitCode=0
Sep 10 19:45:55.777133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:55.777022    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s" event={"ID":"866fb2ea-76c5-48a7-88b4-502beeca87ca","Type":"ContainerDied","Data":"06f81950e5bf0bd32ddb7f02e9ec603185641cbc7115e44315133af98e37e745"}
Sep 10 19:45:56.928513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:56.928484    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"
Sep 10 19:45:56.972790 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:56.972749    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866fb2ea-76c5-48a7-88b4-502beeca87ca-maas-api-service-ca\") pod \"866fb2ea-76c5-48a7-88b4-502beeca87ca\" (UID: \"866fb2ea-76c5-48a7-88b4-502beeca87ca\") "
Sep 10 19:45:56.973207 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:56.973178    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866fb2ea-76c5-48a7-88b4-502beeca87ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "866fb2ea-76c5-48a7-88b4-502beeca87ca" (UID: "866fb2ea-76c5-48a7-88b4-502beeca87ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:57.074464 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:57.074354    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866fb2ea-76c5-48a7-88b4-502beeca87ca-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:45:57.789407 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:57.789377    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"
Sep 10 19:45:57.789601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:57.789419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s" event={"ID":"866fb2ea-76c5-48a7-88b4-502beeca87ca","Type":"ContainerDied","Data":"6c59ef421f5e21f876386688c66c6546609ea41171b20feaeec7f3bfdc3b9a87"}
Sep 10 19:45:57.789601 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:45:57.789467    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c59ef421f5e21f876386688c66c6546609ea41171b20feaeec7f3bfdc3b9a87"
Sep 10 19:46:09.449561 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.449526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:09.479518 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.479479    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"]
Sep 10 19:46:09.479790 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.479737    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd" podUID="7d554a0a-5608-4a46-ae75-6502f7bb99af" containerName="payload-processing" containerID="cri-o://58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d" gracePeriod=30
Sep 10 19:46:09.485338 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.485306    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"]
Sep 10 19:46:09.485580 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.485552    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62" podUID="f4ff7df9-ab24-46c2-88f1-1d0590f136b6" containerName="payload-pre-processing" containerID="cri-o://812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51" gracePeriod=30
Sep 10 19:46:09.550359 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.550323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:09.650694 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.650620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:09.701038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.700942    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"]
Sep 10 19:46:09.702018 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.701988    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d77d5a-nrx6s"]
Sep 10 19:46:09.722827 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.722769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:09.779089 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.779057    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:46:09.782411 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.782385    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:46:09.853550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.853515    2562 generic.go:356] "Generic (PLEG): container finished" podID="f4ff7df9-ab24-46c2-88f1-1d0590f136b6" containerID="812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51" exitCode=0
Sep 10 19:46:09.853725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.853560    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"
Sep 10 19:46:09.853725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.853594    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62" event={"ID":"f4ff7df9-ab24-46c2-88f1-1d0590f136b6","Type":"ContainerDied","Data":"812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51"}
Sep 10 19:46:09.853725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.853648    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62" event={"ID":"f4ff7df9-ab24-46c2-88f1-1d0590f136b6","Type":"ContainerDied","Data":"cb70346cdbadd187997ac5afe1e5adbe18e4a2aee878dddd72e85c614fbfd83b"}
Sep 10 19:46:09.853725 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.853680    2562 scope.go:117] "RemoveContainer" containerID="812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51"
Sep 10 19:46:09.855030 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.855008    2562 generic.go:356] "Generic (PLEG): container finished" podID="7d554a0a-5608-4a46-ae75-6502f7bb99af" containerID="58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d" exitCode=0
Sep 10 19:46:09.855131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.855051    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"
Sep 10 19:46:09.855131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.855065    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd" event={"ID":"7d554a0a-5608-4a46-ae75-6502f7bb99af","Type":"ContainerDied","Data":"58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d"}
Sep 10 19:46:09.855131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.855103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd" event={"ID":"7d554a0a-5608-4a46-ae75-6502f7bb99af","Type":"ContainerDied","Data":"7d35fc1abc88244300c2f22af4c1549925bb9f026b4340bee4223e3254efbe94"}
Sep 10 19:46:09.863065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.863044    2562 scope.go:117] "RemoveContainer" containerID="812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51"
Sep 10 19:46:09.863415 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:46:09.863385    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51\": container with ID starting with 812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51 not found: ID does not exist" containerID="812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51"
Sep 10 19:46:09.863470 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.863427    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51"} err="failed to get container status \"812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51\": rpc error: code = NotFound desc = could not find container \"812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51\": container with ID starting with 812da335ee5224294ea4d1c25ba1ba4100a89d5fa7b1f4c2d93f56816b8d5d51 not found: ID does not exist"
Sep 10 19:46:09.863470 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.863445    2562 scope.go:117] "RemoveContainer" containerID="58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d"
Sep 10 19:46:09.872176 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.872130    2562 scope.go:117] "RemoveContainer" containerID="58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d"
Sep 10 19:46:09.872467 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:46:09.872443    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d\": container with ID starting with 58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d not found: ID does not exist" containerID="58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d"
Sep 10 19:46:09.872543 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.872478    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d"} err="failed to get container status \"58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d\": rpc error: code = NotFound desc = could not find container \"58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d\": container with ID starting with 58751f99555ba486243ac58fba40233ff5d109793c5ce55c092b69dd11e48a6d not found: ID does not exist"
Sep 10 19:46:09.897833 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.897792    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6w5m\" (UniqueName: \"kubernetes.io/projected/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-kube-api-access-f6w5m\") pod \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") "
Sep 10 19:46:09.898031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.897848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-plugins-config-volume\") pod \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\" (UID: \"f4ff7df9-ab24-46c2-88f1-1d0590f136b6\") "
Sep 10 19:46:09.898031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.897881    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d554a0a-5608-4a46-ae75-6502f7bb99af-plugins-config-volume\") pod \"7d554a0a-5608-4a46-ae75-6502f7bb99af\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") "
Sep 10 19:46:09.898031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.897978    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89hgz\" (UniqueName: \"kubernetes.io/projected/7d554a0a-5608-4a46-ae75-6502f7bb99af-kube-api-access-89hgz\") pod \"7d554a0a-5608-4a46-ae75-6502f7bb99af\" (UID: \"7d554a0a-5608-4a46-ae75-6502f7bb99af\") "
Sep 10 19:46:09.898297 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.898269    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4ff7df9-ab24-46c2-88f1-1d0590f136b6" (UID: "f4ff7df9-ab24-46c2-88f1-1d0590f136b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:09.898346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.898315    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d554a0a-5608-4a46-ae75-6502f7bb99af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d554a0a-5608-4a46-ae75-6502f7bb99af" (UID: "7d554a0a-5608-4a46-ae75-6502f7bb99af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:09.898421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.898399    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:09.900046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.900025    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-kube-api-access-f6w5m" (OuterVolumeSpecName: "kube-api-access-f6w5m") pod "f4ff7df9-ab24-46c2-88f1-1d0590f136b6" (UID: "f4ff7df9-ab24-46c2-88f1-1d0590f136b6"). InnerVolumeSpecName "kube-api-access-f6w5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:09.900046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.900041    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d554a0a-5608-4a46-ae75-6502f7bb99af-kube-api-access-89hgz" (OuterVolumeSpecName: "kube-api-access-89hgz") pod "7d554a0a-5608-4a46-ae75-6502f7bb99af" (UID: "7d554a0a-5608-4a46-ae75-6502f7bb99af"). InnerVolumeSpecName "kube-api-access-89hgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:09.999608 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.999513    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d554a0a-5608-4a46-ae75-6502f7bb99af-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:09.999608 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.999547    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89hgz\" (UniqueName: \"kubernetes.io/projected/7d554a0a-5608-4a46-ae75-6502f7bb99af-kube-api-access-89hgz\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:09.999608 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:09.999557    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6w5m\" (UniqueName: \"kubernetes.io/projected/f4ff7df9-ab24-46c2-88f1-1d0590f136b6-kube-api-access-f6w5m\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:10.075492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:10.075455    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:10.184995 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:10.184956    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"]
Sep 10 19:46:10.186919 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:10.186872    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d77d5a-8bdf697b7-q7bfd"]
Sep 10 19:46:10.196552 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:10.196522    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"]
Sep 10 19:46:10.198800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:10.198775    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d77d5a-6b5f9c6598-jtf62"]
Sep 10 19:46:11.695730 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:11.695694    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d554a0a-5608-4a46-ae75-6502f7bb99af" path="/var/lib/kubelet/pods/7d554a0a-5608-4a46-ae75-6502f7bb99af/volumes"
Sep 10 19:46:11.696133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:11.696082    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="866fb2ea-76c5-48a7-88b4-502beeca87ca" path="/var/lib/kubelet/pods/866fb2ea-76c5-48a7-88b4-502beeca87ca/volumes"
Sep 10 19:46:11.696402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:11.696388    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4ff7df9-ab24-46c2-88f1-1d0590f136b6" path="/var/lib/kubelet/pods/f4ff7df9-ab24-46c2-88f1-1d0590f136b6/volumes"
Sep 10 19:46:15.972684 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:15.972639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:16.068625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.068582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:16.213152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.213111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:16.319142 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.319030    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:46:16.558361 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558320    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"]
Sep 10 19:46:16.558768 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558755    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="866fb2ea-76c5-48a7-88b4-502beeca87ca" containerName="revoke-keys"
Sep 10 19:46:16.558828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558770    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="866fb2ea-76c5-48a7-88b4-502beeca87ca" containerName="revoke-keys"
Sep 10 19:46:16.558828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558796    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ff7df9-ab24-46c2-88f1-1d0590f136b6" containerName="payload-pre-processing"
Sep 10 19:46:16.558828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558802    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ff7df9-ab24-46c2-88f1-1d0590f136b6" containerName="payload-pre-processing"
Sep 10 19:46:16.558828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558810    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d554a0a-5608-4a46-ae75-6502f7bb99af" containerName="payload-processing"
Sep 10 19:46:16.558828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558815    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d554a0a-5608-4a46-ae75-6502f7bb99af" containerName="payload-processing"
Sep 10 19:46:16.559011 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558879    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="866fb2ea-76c5-48a7-88b4-502beeca87ca" containerName="revoke-keys"
Sep 10 19:46:16.559011 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558904    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ff7df9-ab24-46c2-88f1-1d0590f136b6" containerName="payload-pre-processing"
Sep 10 19:46:16.559011 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.558912    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d554a0a-5608-4a46-ae75-6502f7bb99af" containerName="payload-processing"
Sep 10 19:46:16.563473 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.563451    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"
Sep 10 19:46:16.566475 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.566454    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:46:16.566556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.566453    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:46:16.570806 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.570738    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"]
Sep 10 19:46:16.667185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.667146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9409ef-76e9-4b34-ab03-3727137161b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-261db5-rdhpn\" (UID: \"7d9409ef-76e9-4b34-ab03-3727137161b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"
Sep 10 19:46:16.681064 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.681026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:16.768313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.768268    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9409ef-76e9-4b34-ab03-3727137161b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-261db5-rdhpn\" (UID: \"7d9409ef-76e9-4b34-ab03-3727137161b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"
Sep 10 19:46:16.769062 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.769040    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9409ef-76e9-4b34-ab03-3727137161b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-261db5-rdhpn\" (UID: \"7d9409ef-76e9-4b34-ab03-3727137161b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"
Sep 10 19:46:16.875726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:16.875645    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"
Sep 10 19:46:17.182502 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:17.182476    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"]
Sep 10 19:46:17.184998 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:46:17.184960    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9409ef_76e9_4b34_ab03_3727137161b5.slice/crio-b872fb3c04b65d09146d37fa5b4648620aa2c2cc3d7df7688f36afb07757b8a9 WatchSource:0}: Error finding container b872fb3c04b65d09146d37fa5b4648620aa2c2cc3d7df7688f36afb07757b8a9: Status 404 returned error can't find the container with id b872fb3c04b65d09146d37fa5b4648620aa2c2cc3d7df7688f36afb07757b8a9
Sep 10 19:46:17.901337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:17.901303    2562 generic.go:356] "Generic (PLEG): container finished" podID="7d9409ef-76e9-4b34-ab03-3727137161b5" containerID="282eed2f47240ffc6296fb4b2f9dcbbac660bdf88882f95c07983ad512cca235" exitCode=0
Sep 10 19:46:17.901513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:17.901389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn" event={"ID":"7d9409ef-76e9-4b34-ab03-3727137161b5","Type":"ContainerDied","Data":"282eed2f47240ffc6296fb4b2f9dcbbac660bdf88882f95c07983ad512cca235"}
Sep 10 19:46:17.901513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:17.901443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn" event={"ID":"7d9409ef-76e9-4b34-ab03-3727137161b5","Type":"ContainerStarted","Data":"b872fb3c04b65d09146d37fa5b4648620aa2c2cc3d7df7688f36afb07757b8a9"}
Sep 10 19:46:19.048502 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:19.048474    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"
Sep 10 19:46:19.193937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:19.193810    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9409ef-76e9-4b34-ab03-3727137161b5-maas-api-service-ca\") pod \"7d9409ef-76e9-4b34-ab03-3727137161b5\" (UID: \"7d9409ef-76e9-4b34-ab03-3727137161b5\") "
Sep 10 19:46:19.194274 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:19.194249    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d9409ef-76e9-4b34-ab03-3727137161b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d9409ef-76e9-4b34-ab03-3727137161b5" (UID: "7d9409ef-76e9-4b34-ab03-3727137161b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:19.294573 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:19.294531    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9409ef-76e9-4b34-ab03-3727137161b5-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:19.913051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:19.913007    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn" event={"ID":"7d9409ef-76e9-4b34-ab03-3727137161b5","Type":"ContainerDied","Data":"b872fb3c04b65d09146d37fa5b4648620aa2c2cc3d7df7688f36afb07757b8a9"}
Sep 10 19:46:19.913051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:19.913042    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"
Sep 10 19:46:19.913256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:19.913070    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b872fb3c04b65d09146d37fa5b4648620aa2c2cc3d7df7688f36afb07757b8a9"
Sep 10 19:46:20.281218 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:20.281175    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"]
Sep 10 19:46:20.329109 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:20.329063    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:21.269053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:21.269014    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"]
Sep 10 19:46:21.295796 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:21.295760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:21.794495 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:21.794454    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.053672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.053589    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"]
Sep 10 19:46:22.054026 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.054012    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d9409ef-76e9-4b34-ab03-3727137161b5" containerName="revoke-keys"
Sep 10 19:46:22.054075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.054027    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d9409ef-76e9-4b34-ab03-3727137161b5" containerName="revoke-keys"
Sep 10 19:46:22.054122 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.054113    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d9409ef-76e9-4b34-ab03-3727137161b5" containerName="revoke-keys"
Sep 10 19:46:22.057277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.057258    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.061198 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.061171    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-836291-openshift-default-dockercfg-p6j4l\""
Sep 10 19:46:22.061330 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.061254    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 19:46:22.073837 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.073805    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"]
Sep 10 19:46:22.088695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.088661    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.223115 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c4a233-f452-477d-87ee-6fef79b90dc9-istiod-ca-cert\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223115 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223119    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-envoy\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223352 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223145    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-data\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223352 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfszb\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-kube-api-access-nfszb\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223352 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-credential-socket\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223352 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223301    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-token\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223352 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-socket\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-certs\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.223535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.223437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-podinfo\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-certs\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324277 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-podinfo\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324320    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c4a233-f452-477d-87ee-6fef79b90dc9-istiod-ca-cert\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324353    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-envoy\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-data\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfszb\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-kube-api-access-nfszb\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-credential-socket\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-token\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324616    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-certs\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-socket\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.324911 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.324744    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-data\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.325265 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.325015    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-socket\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.325265 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.325028    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-credential-socket\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.325265 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.325202    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c4a233-f452-477d-87ee-6fef79b90dc9-istiod-ca-cert\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.326656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.326627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-envoy\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.326882 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.326862    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-podinfo\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.333946 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.333923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-token\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.334220 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.334197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfszb\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-kube-api-access-nfszb\") pod \"e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") " pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.368363 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.368324    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:22.448573 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.448539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:22.655965 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.655933    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"]
Sep 10 19:46:22.657184 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:46:22.657157    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c4a233_f452_477d_87ee_6fef79b90dc9.slice/crio-46fb4186d75d9645d6fd622f9b35e49a14a5ed96215e0366eab33d9210b545de WatchSource:0}: Error finding container 46fb4186d75d9645d6fd622f9b35e49a14a5ed96215e0366eab33d9210b545de: Status 404 returned error can't find the container with id 46fb4186d75d9645d6fd622f9b35e49a14a5ed96215e0366eab33d9210b545de
Sep 10 19:46:22.659633 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.659596    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:22.659728 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.659665    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:22.659728 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.659696    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:22.931760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.931660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" event={"ID":"a0c4a233-f452-477d-87ee-6fef79b90dc9","Type":"ContainerStarted","Data":"07b8b3ed5a31c9fda8b291f5c882086aeff79c80441f5c643d01fbdd5bb23f9d"}
Sep 10 19:46:22.931760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:22.931724    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" event={"ID":"a0c4a233-f452-477d-87ee-6fef79b90dc9","Type":"ContainerStarted","Data":"46fb4186d75d9645d6fd622f9b35e49a14a5ed96215e0366eab33d9210b545de"}
Sep 10 19:46:23.369126 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:23.369040    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:23.375643 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:23.375613    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:23.400337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:23.400276    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podStartSLOduration=1.400256985 podStartE2EDuration="1.400256985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:22.969618454 +0000 UTC m=+1575.909652297" watchObservedRunningTime="2026-09-10 19:46:23.400256985 +0000 UTC m=+1576.340290818"
Sep 10 19:46:23.937604 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:23.937569    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:23.938757 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:23.938733    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:24.893228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:24.893179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:25.009447 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:25.009409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:26.063842 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.063807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:26.164073 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.164042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:26.463526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.463493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:26.584760 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.584712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:26.685108 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.685066    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"]
Sep 10 19:46:26.686266 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.686228    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:26.690301 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.690276    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:26.693709 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.693684    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-836291\""
Sep 10 19:46:26.693960 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.693941    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"]
Sep 10 19:46:26.694392 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.694374    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-836291-dockercfg-4czst\""
Sep 10 19:46:26.697445 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.697423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:26.704420 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.704393    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"]
Sep 10 19:46:26.721522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.721437    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"]
Sep 10 19:46:26.871781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.871739    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhng4\" (UniqueName: \"kubernetes.io/projected/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-kube-api-access-fhng4\") pod \"payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:26.872013 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.871809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:26.872013 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.871933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmmbn\" (UniqueName: \"kubernetes.io/projected/9081bfa1-158d-4993-8319-24bc4991513b-kube-api-access-qmmbn\") pod \"payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:26.872124 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.872107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9081bfa1-158d-4993-8319-24bc4991513b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:26.973422 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.973332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmmbn\" (UniqueName: \"kubernetes.io/projected/9081bfa1-158d-4993-8319-24bc4991513b-kube-api-access-qmmbn\") pod \"payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:26.973571 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.973422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9081bfa1-158d-4993-8319-24bc4991513b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:26.973571 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.973450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhng4\" (UniqueName: \"kubernetes.io/projected/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-kube-api-access-fhng4\") pod \"payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:26.973571 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.973481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:26.974249 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.974229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:26.974301 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.974241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9081bfa1-158d-4993-8319-24bc4991513b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:26.990775 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.990674    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhng4\" (UniqueName: \"kubernetes.io/projected/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-kube-api-access-fhng4\") pod \"payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:26.997032 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:26.991678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmmbn\" (UniqueName: \"kubernetes.io/projected/9081bfa1-158d-4993-8319-24bc4991513b-kube-api-access-qmmbn\") pod \"payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:27.002515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.002485    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:27.011469 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.011442    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:27.284766 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.284742    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"]
Sep 10 19:46:27.287297 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:46:27.287269    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9081bfa1_158d_4993_8319_24bc4991513b.slice/crio-88dbeaa4e9d3d19ce7eaf1a73d60054d2625dd76699e571b7b3b6e0fbf299b82 WatchSource:0}: Error finding container 88dbeaa4e9d3d19ce7eaf1a73d60054d2625dd76699e571b7b3b6e0fbf299b82: Status 404 returned error can't find the container with id 88dbeaa4e9d3d19ce7eaf1a73d60054d2625dd76699e571b7b3b6e0fbf299b82
Sep 10 19:46:27.291458 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.291434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"]
Sep 10 19:46:27.292727 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:46:27.292697    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac23ee40_e7ea_4d50_aaf2_2e2193c9bfa2.slice/crio-c282c23270c8864e8a07072a0c76b462bbc9f2f758b52671cac3a4731e1bdc20 WatchSource:0}: Error finding container c282c23270c8864e8a07072a0c76b462bbc9f2f758b52671cac3a4731e1bdc20: Status 404 returned error can't find the container with id c282c23270c8864e8a07072a0c76b462bbc9f2f758b52671cac3a4731e1bdc20
Sep 10 19:46:27.961785 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.961744    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8" event={"ID":"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2","Type":"ContainerStarted","Data":"2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8"}
Sep 10 19:46:27.962049 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.961812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8" event={"ID":"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2","Type":"ContainerStarted","Data":"c282c23270c8864e8a07072a0c76b462bbc9f2f758b52671cac3a4731e1bdc20"}
Sep 10 19:46:27.962049 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.961931    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:27.963740 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.963708    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf" event={"ID":"9081bfa1-158d-4993-8319-24bc4991513b","Type":"ContainerStarted","Data":"36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18"}
Sep 10 19:46:27.963873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.963762    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf" event={"ID":"9081bfa1-158d-4993-8319-24bc4991513b","Type":"ContainerStarted","Data":"88dbeaa4e9d3d19ce7eaf1a73d60054d2625dd76699e571b7b3b6e0fbf299b82"}
Sep 10 19:46:27.963873 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.963790    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:27.986515 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:27.986454    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8" podStartSLOduration=1.986433511 podStartE2EDuration="1.986433511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:27.9815064 +0000 UTC m=+1580.921540233" watchObservedRunningTime="2026-09-10 19:46:27.986433511 +0000 UTC m=+1580.926467341"
Sep 10 19:46:28.002305 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:28.002247    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf" podStartSLOduration=2.002229343 podStartE2EDuration="2.002229343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:28.000831675 +0000 UTC m=+1580.940865565" watchObservedRunningTime="2026-09-10 19:46:28.002229343 +0000 UTC m=+1580.942263175"
Sep 10 19:46:31.897534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:31.897480    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:31.928487 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:31.928451    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"]
Sep 10 19:46:31.928771 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:31.928733    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs" podUID="7a26c900-3b04-40d1-a6fc-c1df4a12eea9" containerName="payload-pre-processing" containerID="cri-o://5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01" gracePeriod=30
Sep 10 19:46:31.997759 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:31.997716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:32.095817 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.095778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:32.114709 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.114673    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"]
Sep 10 19:46:32.116846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.116817    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-261db5-rdhpn"]
Sep 10 19:46:32.196572 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.196537    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:46:32.210567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.210534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:32.231731 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.231701    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-plugins-config-volume\") pod \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") "
Sep 10 19:46:32.231944 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.231738    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4hqd\" (UniqueName: \"kubernetes.io/projected/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-kube-api-access-r4hqd\") pod \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\" (UID: \"7a26c900-3b04-40d1-a6fc-c1df4a12eea9\") "
Sep 10 19:46:32.232114 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.232090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a26c900-3b04-40d1-a6fc-c1df4a12eea9" (UID: "7a26c900-3b04-40d1-a6fc-c1df4a12eea9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:32.232204 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.232186    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:32.233852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.233823    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-kube-api-access-r4hqd" (OuterVolumeSpecName: "kube-api-access-r4hqd") pod "7a26c900-3b04-40d1-a6fc-c1df4a12eea9" (UID: "7a26c900-3b04-40d1-a6fc-c1df4a12eea9"). InnerVolumeSpecName "kube-api-access-r4hqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:32.333286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.333248    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4hqd\" (UniqueName: \"kubernetes.io/projected/7a26c900-3b04-40d1-a6fc-c1df4a12eea9-kube-api-access-r4hqd\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:32.510783 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.510693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:32.714466 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.714431    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:46:32.995995 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.995953    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a26c900-3b04-40d1-a6fc-c1df4a12eea9" containerID="5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01" exitCode=0
Sep 10 19:46:32.996479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.996003    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"
Sep 10 19:46:32.996479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.996015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs" event={"ID":"7a26c900-3b04-40d1-a6fc-c1df4a12eea9","Type":"ContainerDied","Data":"5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01"}
Sep 10 19:46:32.996479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.996071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs" event={"ID":"7a26c900-3b04-40d1-a6fc-c1df4a12eea9","Type":"ContainerDied","Data":"ad027e03d0c127577095206b3e3db13907bb9800ed2057c55d61b29b9a607942"}
Sep 10 19:46:32.996479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:32.996105    2562 scope.go:117] "RemoveContainer" containerID="5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01"
Sep 10 19:46:33.006937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:33.006915    2562 scope.go:117] "RemoveContainer" containerID="5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01"
Sep 10 19:46:33.007285 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:46:33.007262    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01\": container with ID starting with 5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01 not found: ID does not exist" containerID="5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01"
Sep 10 19:46:33.007384 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:33.007293    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01"} err="failed to get container status \"5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01\": rpc error: code = NotFound desc = could not find container \"5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01\": container with ID starting with 5307b3b44b304b96b176165203be1c89fc789755cd2634697a5b9bfa861ffb01 not found: ID does not exist"
Sep 10 19:46:33.018845 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:33.018814    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"]
Sep 10 19:46:33.025712 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:33.025679    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-261db5-67db944b9d-qz7xs"]
Sep 10 19:46:33.696437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:33.696399    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a26c900-3b04-40d1-a6fc-c1df4a12eea9" path="/var/lib/kubelet/pods/7a26c900-3b04-40d1-a6fc-c1df4a12eea9/volumes"
Sep 10 19:46:33.696784 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:33.696770    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d9409ef-76e9-4b34-ab03-3727137161b5" path="/var/lib/kubelet/pods/7d9409ef-76e9-4b34-ab03-3727137161b5/volumes"
Sep 10 19:46:35.301419 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.301364    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx" podUID="98b8658c-6703-495f-a213-f1c600d37351" containerName="main" containerID="cri-o://2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa" gracePeriod=45
Sep 10 19:46:35.592068 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.592041    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:46:35.671516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671477    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b8658c-6703-495f-a213-f1c600d37351-tls-certs\") pod \"98b8658c-6703-495f-a213-f1c600d37351\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") "
Sep 10 19:46:35.671516 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671515    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-model-cache\") pod \"98b8658c-6703-495f-a213-f1c600d37351\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") "
Sep 10 19:46:35.671785 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671570    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-home\") pod \"98b8658c-6703-495f-a213-f1c600d37351\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") "
Sep 10 19:46:35.671785 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671622    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-tmp-dir\") pod \"98b8658c-6703-495f-a213-f1c600d37351\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") "
Sep 10 19:46:35.671785 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671690    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp6s5\" (UniqueName: \"kubernetes.io/projected/98b8658c-6703-495f-a213-f1c600d37351-kube-api-access-dp6s5\") pod \"98b8658c-6703-495f-a213-f1c600d37351\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") "
Sep 10 19:46:35.671785 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671731    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-dshm\") pod \"98b8658c-6703-495f-a213-f1c600d37351\" (UID: \"98b8658c-6703-495f-a213-f1c600d37351\") "
Sep 10 19:46:35.671785 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671752    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-model-cache" (OuterVolumeSpecName: "model-cache") pod "98b8658c-6703-495f-a213-f1c600d37351" (UID: "98b8658c-6703-495f-a213-f1c600d37351"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:35.672029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671850    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98b8658c-6703-495f-a213-f1c600d37351" (UID: "98b8658c-6703-495f-a213-f1c600d37351"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:35.672029 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.671869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-home" (OuterVolumeSpecName: "home") pod "98b8658c-6703-495f-a213-f1c600d37351" (UID: "98b8658c-6703-495f-a213-f1c600d37351"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:35.672151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.672071    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.672151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.672089    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.672151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.672101    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.673731 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.673704    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98b8658c-6703-495f-a213-f1c600d37351-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98b8658c-6703-495f-a213-f1c600d37351" (UID: "98b8658c-6703-495f-a213-f1c600d37351"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:35.673861 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.673785    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b8658c-6703-495f-a213-f1c600d37351-kube-api-access-dp6s5" (OuterVolumeSpecName: "kube-api-access-dp6s5") pod "98b8658c-6703-495f-a213-f1c600d37351" (UID: "98b8658c-6703-495f-a213-f1c600d37351"). InnerVolumeSpecName "kube-api-access-dp6s5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:35.673861 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.673793    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-dshm" (OuterVolumeSpecName: "dshm") pod "98b8658c-6703-495f-a213-f1c600d37351" (UID: "98b8658c-6703-495f-a213-f1c600d37351"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:35.773090 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.773048    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp6s5\" (UniqueName: \"kubernetes.io/projected/98b8658c-6703-495f-a213-f1c600d37351-kube-api-access-dp6s5\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.773090 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.773082    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b8658c-6703-495f-a213-f1c600d37351-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:35.773090 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:35.773094    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b8658c-6703-495f-a213-f1c600d37351-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:36.014296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.014260    2562 generic.go:356] "Generic (PLEG): container finished" podID="98b8658c-6703-495f-a213-f1c600d37351" containerID="2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa" exitCode=0
Sep 10 19:46:36.014456 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.014307    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"
Sep 10 19:46:36.014456 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.014317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx" event={"ID":"98b8658c-6703-495f-a213-f1c600d37351","Type":"ContainerDied","Data":"2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa"}
Sep 10 19:46:36.014456 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.014349    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx" event={"ID":"98b8658c-6703-495f-a213-f1c600d37351","Type":"ContainerDied","Data":"831f86d2be706a90f9d4d46b5b9d48299ad020115778589b5f0dcc5448d0af8f"}
Sep 10 19:46:36.014456 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.014366    2562 scope.go:117] "RemoveContainer" containerID="2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa"
Sep 10 19:46:36.023429 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.023407    2562 scope.go:117] "RemoveContainer" containerID="2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa"
Sep 10 19:46:36.023715 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:46:36.023697    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa\": container with ID starting with 2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa not found: ID does not exist" containerID="2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa"
Sep 10 19:46:36.023765 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.023741    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa"} err="failed to get container status \"2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa\": rpc error: code = NotFound desc = could not find container \"2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa\": container with ID starting with 2013a8ebdda4efcf7be776a4708eb1bfbdadfa0cfb734fa7fd3c6f8da49166fa not found: ID does not exist"
Sep 10 19:46:36.033555 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.033524    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"]
Sep 10 19:46:36.038270 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.038243    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx"]
Sep 10 19:46:36.289137 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.289043    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt" podUID="439e5132-a506-4fb5-9479-5ad25e76d5ed" containerName="main" containerID="cri-o://7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684" gracePeriod=45
Sep 10 19:46:36.580392 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.580365    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:46:36.682053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682013    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlwvw\" (UniqueName: \"kubernetes.io/projected/439e5132-a506-4fb5-9479-5ad25e76d5ed-kube-api-access-nlwvw\") pod \"439e5132-a506-4fb5-9479-5ad25e76d5ed\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") "
Sep 10 19:46:36.682053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682057    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-tmp-dir\") pod \"439e5132-a506-4fb5-9479-5ad25e76d5ed\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") "
Sep 10 19:46:36.682328 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682074    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-home\") pod \"439e5132-a506-4fb5-9479-5ad25e76d5ed\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") "
Sep 10 19:46:36.682328 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682140    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-dshm\") pod \"439e5132-a506-4fb5-9479-5ad25e76d5ed\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") "
Sep 10 19:46:36.682328 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682168    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/439e5132-a506-4fb5-9479-5ad25e76d5ed-tls-certs\") pod \"439e5132-a506-4fb5-9479-5ad25e76d5ed\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") "
Sep 10 19:46:36.682328 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682202    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-model-cache\") pod \"439e5132-a506-4fb5-9479-5ad25e76d5ed\" (UID: \"439e5132-a506-4fb5-9479-5ad25e76d5ed\") "
Sep 10 19:46:36.682526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682347    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "439e5132-a506-4fb5-9479-5ad25e76d5ed" (UID: "439e5132-a506-4fb5-9479-5ad25e76d5ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:36.682526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682356    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-home" (OuterVolumeSpecName: "home") pod "439e5132-a506-4fb5-9479-5ad25e76d5ed" (UID: "439e5132-a506-4fb5-9479-5ad25e76d5ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:36.682526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682484    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "439e5132-a506-4fb5-9479-5ad25e76d5ed" (UID: "439e5132-a506-4fb5-9479-5ad25e76d5ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:36.682672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682587    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:36.682672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682605    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:36.682672 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.682618    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:36.684522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.684493    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-dshm" (OuterVolumeSpecName: "dshm") pod "439e5132-a506-4fb5-9479-5ad25e76d5ed" (UID: "439e5132-a506-4fb5-9479-5ad25e76d5ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:46:36.684644 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.684528    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/439e5132-a506-4fb5-9479-5ad25e76d5ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "439e5132-a506-4fb5-9479-5ad25e76d5ed" (UID: "439e5132-a506-4fb5-9479-5ad25e76d5ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:36.684644 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.684577    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439e5132-a506-4fb5-9479-5ad25e76d5ed-kube-api-access-nlwvw" (OuterVolumeSpecName: "kube-api-access-nlwvw") pod "439e5132-a506-4fb5-9479-5ad25e76d5ed" (UID: "439e5132-a506-4fb5-9479-5ad25e76d5ed"). InnerVolumeSpecName "kube-api-access-nlwvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:36.783408 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.783366    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlwvw\" (UniqueName: \"kubernetes.io/projected/439e5132-a506-4fb5-9479-5ad25e76d5ed-kube-api-access-nlwvw\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:36.783408 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.783403    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/439e5132-a506-4fb5-9479-5ad25e76d5ed-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:36.783408 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:36.783413    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/439e5132-a506-4fb5-9479-5ad25e76d5ed-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:37.022576 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.022532    2562 generic.go:356] "Generic (PLEG): container finished" podID="439e5132-a506-4fb5-9479-5ad25e76d5ed" containerID="7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684" exitCode=0
Sep 10 19:46:37.022761 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.022603    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"
Sep 10 19:46:37.022761 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.022620    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt" event={"ID":"439e5132-a506-4fb5-9479-5ad25e76d5ed","Type":"ContainerDied","Data":"7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684"}
Sep 10 19:46:37.022761 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.022669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt" event={"ID":"439e5132-a506-4fb5-9479-5ad25e76d5ed","Type":"ContainerDied","Data":"37169f39c9b37277807dda5653ae92d0d46099498f43394f50ef60adb70333ce"}
Sep 10 19:46:37.022761 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.022689    2562 scope.go:117] "RemoveContainer" containerID="7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684"
Sep 10 19:46:37.032304 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.032286    2562 scope.go:117] "RemoveContainer" containerID="7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684"
Sep 10 19:46:37.032668 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:46:37.032645    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684\": container with ID starting with 7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684 not found: ID does not exist" containerID="7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684"
Sep 10 19:46:37.032712 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.032679    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684"} err="failed to get container status \"7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684\": rpc error: code = NotFound desc = could not find container \"7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684\": container with ID starting with 7fcc13017deae59d41cadd54fb16183b1f0beb8e68afdf7e78af14f119309684 not found: ID does not exist"
Sep 10 19:46:37.062028 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.061981    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"]
Sep 10 19:46:37.067586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.067545    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2a1fa9/rate-test-model-2a1fa9-kserve-f74687999-vh4nt"]
Sep 10 19:46:37.698272 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.698227    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="439e5132-a506-4fb5-9479-5ad25e76d5ed" path="/var/lib/kubelet/pods/439e5132-a506-4fb5-9479-5ad25e76d5ed/volumes"
Sep 10 19:46:37.698805 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:37.698780    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b8658c-6703-495f-a213-f1c600d37351" path="/var/lib/kubelet/pods/98b8658c-6703-495f-a213-f1c600d37351/volumes"
Sep 10 19:46:38.512316 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:38.512268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:38.584436 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:38.584395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:38.869383 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:38.869278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:38.971883 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:38.971853    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:46:38.972077 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:38.971959    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:46:39.166023 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.165973    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"]
Sep 10 19:46:39.166681 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166659    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a26c900-3b04-40d1-a6fc-c1df4a12eea9" containerName="payload-pre-processing"
Sep 10 19:46:39.166746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166687    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a26c900-3b04-40d1-a6fc-c1df4a12eea9" containerName="payload-pre-processing"
Sep 10 19:46:39.166746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166719    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b8658c-6703-495f-a213-f1c600d37351" containerName="main"
Sep 10 19:46:39.166746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166729    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b8658c-6703-495f-a213-f1c600d37351" containerName="main"
Sep 10 19:46:39.166746 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166742    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="439e5132-a506-4fb5-9479-5ad25e76d5ed" containerName="main"
Sep 10 19:46:39.166871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166751    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="439e5132-a506-4fb5-9479-5ad25e76d5ed" containerName="main"
Sep 10 19:46:39.166871 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166866    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="439e5132-a506-4fb5-9479-5ad25e76d5ed" containerName="main"
Sep 10 19:46:39.166988 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166879    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b8658c-6703-495f-a213-f1c600d37351" containerName="main"
Sep 10 19:46:39.166988 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.166919    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a26c900-3b04-40d1-a6fc-c1df4a12eea9" containerName="payload-pre-processing"
Sep 10 19:46:39.170599 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.170573    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"
Sep 10 19:46:39.173619 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.173592    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"]
Sep 10 19:46:39.173769 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.173626    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:46:39.174098 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.174075    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:46:39.208555 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.208515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ee8044-6c00-4064-9565-37fe14f7c01f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf\" (UID: \"a9ee8044-6c00-4064-9565-37fe14f7c01f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"
Sep 10 19:46:39.281025 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.280980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:39.309070 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.309038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ee8044-6c00-4064-9565-37fe14f7c01f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf\" (UID: \"a9ee8044-6c00-4064-9565-37fe14f7c01f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"
Sep 10 19:46:39.309742 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.309716    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ee8044-6c00-4064-9565-37fe14f7c01f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf\" (UID: \"a9ee8044-6c00-4064-9565-37fe14f7c01f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"
Sep 10 19:46:39.485664 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.485535    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"
Sep 10 19:46:39.764080 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:39.764051    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"]
Sep 10 19:46:39.766193 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:46:39.766158    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ee8044_6c00_4064_9565_37fe14f7c01f.slice/crio-ad87f04c35bf5dd9ba795ec07675d3502e99562bbb5e6a4c65fd4f9e477f5e5e WatchSource:0}: Error finding container ad87f04c35bf5dd9ba795ec07675d3502e99562bbb5e6a4c65fd4f9e477f5e5e: Status 404 returned error can't find the container with id ad87f04c35bf5dd9ba795ec07675d3502e99562bbb5e6a4c65fd4f9e477f5e5e
Sep 10 19:46:40.045930 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:40.045868    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf" event={"ID":"a9ee8044-6c00-4064-9565-37fe14f7c01f","Type":"ContainerStarted","Data":"ad87f04c35bf5dd9ba795ec07675d3502e99562bbb5e6a4c65fd4f9e477f5e5e"}
Sep 10 19:46:40.548561 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:40.548510    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w3-07c5fe/rate-test-model-07c5fe-kserve-55bb6df899-8hfvx" podUID="98b8658c-6703-495f-a213-f1c600d37351" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:46:41.053505 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:41.053462    2562 generic.go:356] "Generic (PLEG): container finished" podID="a9ee8044-6c00-4064-9565-37fe14f7c01f" containerID="49539beb4afc5b8fcaf372bae6f6581573a406ad9fdb32358cea0e961ca44fb1" exitCode=0
Sep 10 19:46:41.053882 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:41.053547    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf" event={"ID":"a9ee8044-6c00-4064-9565-37fe14f7c01f","Type":"ContainerDied","Data":"49539beb4afc5b8fcaf372bae6f6581573a406ad9fdb32358cea0e961ca44fb1"}
Sep 10 19:46:42.210756 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:42.210724    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"
Sep 10 19:46:42.237185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:42.237152    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ee8044-6c00-4064-9565-37fe14f7c01f-maas-api-service-ca\") pod \"a9ee8044-6c00-4064-9565-37fe14f7c01f\" (UID: \"a9ee8044-6c00-4064-9565-37fe14f7c01f\") "
Sep 10 19:46:42.237549 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:42.237525    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ee8044-6c00-4064-9565-37fe14f7c01f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9ee8044-6c00-4064-9565-37fe14f7c01f" (UID: "a9ee8044-6c00-4064-9565-37fe14f7c01f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:42.338941 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:42.338808    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ee8044-6c00-4064-9565-37fe14f7c01f-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:43.066801 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.066765    2562 generic.go:356] "Generic (PLEG): container finished" podID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerID="07b8b3ed5a31c9fda8b291f5c882086aeff79c80441f5c643d01fbdd5bb23f9d" exitCode=0
Sep 10 19:46:43.067052 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.066837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" event={"ID":"a0c4a233-f452-477d-87ee-6fef79b90dc9","Type":"ContainerDied","Data":"07b8b3ed5a31c9fda8b291f5c882086aeff79c80441f5c643d01fbdd5bb23f9d"}
Sep 10 19:46:43.067433 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.067416    2562 scope.go:117] "RemoveContainer" containerID="07b8b3ed5a31c9fda8b291f5c882086aeff79c80441f5c643d01fbdd5bb23f9d"
Sep 10 19:46:43.068422 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.068396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf" event={"ID":"a9ee8044-6c00-4064-9565-37fe14f7c01f","Type":"ContainerDied","Data":"ad87f04c35bf5dd9ba795ec07675d3502e99562bbb5e6a4c65fd4f9e477f5e5e"}
Sep 10 19:46:43.068631 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.068426    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"
Sep 10 19:46:43.068631 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.068437    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad87f04c35bf5dd9ba795ec07675d3502e99562bbb5e6a4c65fd4f9e477f5e5e"
Sep 10 19:46:43.068757 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.068721    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:43.068878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.068857    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:43.068950 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:43.068927    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:44.077484 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:44.077449    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" event={"ID":"a0c4a233-f452-477d-87ee-6fef79b90dc9","Type":"ContainerStarted","Data":"5555a3c0da59f031e9cc61115ec4f2c0027f2781de5c6441a1f8ceea6c3806bb"}
Sep 10 19:46:44.107615 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:44.107578    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:46:44.369553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:44.369442    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:44.369772 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:44.369748    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 10 19:46:44.369855 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:44.369802    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 10 19:46:45.369088 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:45.369044    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 10 19:46:45.369494 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:45.369109    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 10 19:46:46.369622 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:46.369571    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 10 19:46:46.370195 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:46.369639    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 10 19:46:47.368795 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:47.368756    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 10 19:46:47.369039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:47.368823    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 10 19:46:47.393219 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:47.393180    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:48.369316 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:48.369271    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 10 19:46:48.369512 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:48.369347    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 10 19:46:49.220010 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.219970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:49.221100 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.221078    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"]
Sep 10 19:46:49.230669 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.230634    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ecadb8-29817825-sb8n6"]
Sep 10 19:46:49.274163 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.274123    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"]
Sep 10 19:46:49.274479 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.274422    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx" podUID="7afec082-517e-431d-8f16-41bb6b086b88" containerName="payload-processing" containerID="cri-o://b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17" gracePeriod=30
Sep 10 19:46:49.324808 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.324751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:49.369829 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.369783    2562 patch_prober.go:28] interesting pod/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 10 19:46:49.370047 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.369857    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 10 19:46:49.429768 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.429730    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:49.508690 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.508602    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"]
Sep 10 19:46:49.513219 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.513189    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ecadb8-pctvf"]
Sep 10 19:46:49.516391 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.516360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:49.542815 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.542781    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:46:49.614681 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.614643    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whhm4\" (UniqueName: \"kubernetes.io/projected/7afec082-517e-431d-8f16-41bb6b086b88-kube-api-access-whhm4\") pod \"7afec082-517e-431d-8f16-41bb6b086b88\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") "
Sep 10 19:46:49.614898 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.614719    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7afec082-517e-431d-8f16-41bb6b086b88-plugins-config-volume\") pod \"7afec082-517e-431d-8f16-41bb6b086b88\" (UID: \"7afec082-517e-431d-8f16-41bb6b086b88\") "
Sep 10 19:46:49.615132 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.615108    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7afec082-517e-431d-8f16-41bb6b086b88-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7afec082-517e-431d-8f16-41bb6b086b88" (UID: "7afec082-517e-431d-8f16-41bb6b086b88"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:49.616898 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.616870    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7afec082-517e-431d-8f16-41bb6b086b88-kube-api-access-whhm4" (OuterVolumeSpecName: "kube-api-access-whhm4") pod "7afec082-517e-431d-8f16-41bb6b086b88" (UID: "7afec082-517e-431d-8f16-41bb6b086b88"). InnerVolumeSpecName "kube-api-access-whhm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:49.695916 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.695866    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce7493a-4880-48da-810e-5220ad4c5af8" path="/var/lib/kubelet/pods/7ce7493a-4880-48da-810e-5220ad4c5af8/volumes"
Sep 10 19:46:49.696256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.696242    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9ee8044-6c00-4064-9565-37fe14f7c01f" path="/var/lib/kubelet/pods/a9ee8044-6c00-4064-9565-37fe14f7c01f/volumes"
Sep 10 19:46:49.716194 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.716150    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whhm4\" (UniqueName: \"kubernetes.io/projected/7afec082-517e-431d-8f16-41bb6b086b88-kube-api-access-whhm4\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:49.716194 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.716179    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7afec082-517e-431d-8f16-41bb6b086b88-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:46:49.893361 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.893314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:49.992519 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:49.992482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:50.111336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.111300    2562 generic.go:356] "Generic (PLEG): container finished" podID="7afec082-517e-431d-8f16-41bb6b086b88" containerID="b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17" exitCode=0
Sep 10 19:46:50.111519 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.111343    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"
Sep 10 19:46:50.111519 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.111385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx" event={"ID":"7afec082-517e-431d-8f16-41bb6b086b88","Type":"ContainerDied","Data":"b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17"}
Sep 10 19:46:50.111519 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.111437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx" event={"ID":"7afec082-517e-431d-8f16-41bb6b086b88","Type":"ContainerDied","Data":"ff74ca10c77edb32a5dbed2f121606898ac20e9c4dbef061ab8a37ed02914fad"}
Sep 10 19:46:50.111519 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.111469    2562 scope.go:117] "RemoveContainer" containerID="b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17"
Sep 10 19:46:50.120828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.120810    2562 scope.go:117] "RemoveContainer" containerID="b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17"
Sep 10 19:46:50.121202 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:46:50.121177    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17\": container with ID starting with b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17 not found: ID does not exist" containerID="b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17"
Sep 10 19:46:50.121304 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.121209    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17"} err="failed to get container status \"b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17\": rpc error: code = NotFound desc = could not find container \"b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17\": container with ID starting with b32e71915a2c28d15f23b2b4a33c5b1dc0e451661b433f4b66cdcab232714b17 not found: ID does not exist"
Sep 10 19:46:50.130364 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.130331    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"]
Sep 10 19:46:50.135949 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.135923    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ecadb8-87cb58d5f-n66rx"]
Sep 10 19:46:50.372938 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.372843    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:50.373425 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.373404    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:50.374044 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.374023    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:46:50.581134 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.581092    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:50.878994 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.878854    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"]
Sep 10 19:46:50.879379 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.879361    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9ee8044-6c00-4064-9565-37fe14f7c01f" containerName="revoke-keys"
Sep 10 19:46:50.879435 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.879381    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9ee8044-6c00-4064-9565-37fe14f7c01f" containerName="revoke-keys"
Sep 10 19:46:50.879435 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.879393    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7afec082-517e-431d-8f16-41bb6b086b88" containerName="payload-processing"
Sep 10 19:46:50.879435 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.879399    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7afec082-517e-431d-8f16-41bb6b086b88" containerName="payload-processing"
Sep 10 19:46:50.879544 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.879471    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9ee8044-6c00-4064-9565-37fe14f7c01f" containerName="revoke-keys"
Sep 10 19:46:50.879544 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.879479    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7afec082-517e-431d-8f16-41bb6b086b88" containerName="payload-processing"
Sep 10 19:46:50.884230 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.884202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.885447 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.885420    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"]
Sep 10 19:46:50.887130 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.887110    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-b588eb-openshift-default-dockercfg-ps5bw\""
Sep 10 19:46:50.911019 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.910979    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:50.928306 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bcpj\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-kube-api-access-8bcpj\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-podinfo\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928492 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-envoy\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928614 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-data\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928614 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-credential-socket\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928614 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-certs\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928618    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-socket\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a4a27c-99fb-48cb-9f93-996855f036bd-istiod-ca-cert\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:50.928758 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:50.928731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-token\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.029655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.029616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-data\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.029655 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.029657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-credential-socket\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.029961 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.029677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-certs\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.029961 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.029809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-socket\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.029961 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.029854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a4a27c-99fb-48cb-9f93-996855f036bd-istiod-ca-cert\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.029965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-token\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bcpj\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-kube-api-access-8bcpj\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-certs\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-podinfo\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-data\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-envoy\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030496 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-credential-socket\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030496 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-socket\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.030568 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.030555    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a4a27c-99fb-48cb-9f93-996855f036bd-istiod-ca-cert\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.032262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.032241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-envoy\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.032520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.032504    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-podinfo\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.046629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.046598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bcpj\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-kube-api-access-8bcpj\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.048089 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.048069    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-token\") pod \"e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") " pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.196401 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.196358    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:51.258038 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.257933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:51.476262 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.476234    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"]
Sep 10 19:46:51.477955 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:46:51.477925    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a4a27c_99fb_48cb_9f93_996855f036bd.slice/crio-e2921b21f0779f246a44d80556971d019d6a54e10b72faa802d443d643003b21 WatchSource:0}: Error finding container e2921b21f0779f246a44d80556971d019d6a54e10b72faa802d443d643003b21: Status 404 returned error can't find the container with id e2921b21f0779f246a44d80556971d019d6a54e10b72faa802d443d643003b21
Sep 10 19:46:51.480151 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.480117    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:51.480211 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.480186    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:51.480256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.480213    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:46:51.698053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:51.698013    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7afec082-517e-431d-8f16-41bb6b086b88" path="/var/lib/kubelet/pods/7afec082-517e-431d-8f16-41bb6b086b88/volumes"
Sep 10 19:46:52.125509 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:52.125472    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" event={"ID":"11a4a27c-99fb-48cb-9f93-996855f036bd","Type":"ContainerStarted","Data":"11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8"}
Sep 10 19:46:52.125724 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:52.125533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" event={"ID":"11a4a27c-99fb-48cb-9f93-996855f036bd","Type":"ContainerStarted","Data":"e2921b21f0779f246a44d80556971d019d6a54e10b72faa802d443d643003b21"}
Sep 10 19:46:52.149521 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:52.149465    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" podStartSLOduration=2.149447471 podStartE2EDuration="2.149447471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:52.148438605 +0000 UTC m=+1605.088472439" watchObservedRunningTime="2026-09-10 19:46:52.149447471 +0000 UTC m=+1605.089481300"
Sep 10 19:46:52.197083 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:52.197041    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:52.202797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:52.202768    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:53.131722 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:53.131692    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:53.133296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:53.133273    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:46:54.107623 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:54.107569    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:54.192069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:54.192017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:54.729027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:54.728982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:54.948600 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:54.948562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:55.210745 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:55.210704    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:46:55.767298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:55.767260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:57.165419 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.165385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:57.331812 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.331768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:57.653392 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.653355    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"]
Sep 10 19:46:57.658441 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.658418    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"
Sep 10 19:46:57.661005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.660982    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:46:57.661985 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.661961    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:46:57.668527 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.668499    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"]
Sep 10 19:46:57.799715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.799673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38b6cfe-6e80-4ebd-afd2-fb972fb14f27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp\" (UID: \"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"
Sep 10 19:46:57.899476 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.899439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:46:57.900371 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.900344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38b6cfe-6e80-4ebd-afd2-fb972fb14f27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp\" (UID: \"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"
Sep 10 19:46:57.900973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.900953    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38b6cfe-6e80-4ebd-afd2-fb972fb14f27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp\" (UID: \"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"
Sep 10 19:46:57.971797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:57.971702    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"
Sep 10 19:46:58.249532 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:58.249505    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"]
Sep 10 19:46:58.251935 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:46:58.251881    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38b6cfe_6e80_4ebd_afd2_fb972fb14f27.slice/crio-3faac2b35bcb03a55d99ec22e0025c8bdf512c52aec81da0ce734f8d79790633 WatchSource:0}: Error finding container 3faac2b35bcb03a55d99ec22e0025c8bdf512c52aec81da0ce734f8d79790633: Status 404 returned error can't find the container with id 3faac2b35bcb03a55d99ec22e0025c8bdf512c52aec81da0ce734f8d79790633
Sep 10 19:46:59.166226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:59.166184    2562 generic.go:356] "Generic (PLEG): container finished" podID="f38b6cfe-6e80-4ebd-afd2-fb972fb14f27" containerID="52436710a11d92761130c1ac7c74446cd372e87c51efc0ed7800b13d83b3c7d2" exitCode=0
Sep 10 19:46:59.166417 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:59.166268    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp" event={"ID":"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27","Type":"ContainerDied","Data":"52436710a11d92761130c1ac7c74446cd372e87c51efc0ed7800b13d83b3c7d2"}
Sep 10 19:46:59.166417 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:46:59.166333    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp" event={"ID":"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27","Type":"ContainerStarted","Data":"3faac2b35bcb03a55d99ec22e0025c8bdf512c52aec81da0ce734f8d79790633"}
Sep 10 19:47:00.308437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:00.308410    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"
Sep 10 19:47:00.427476 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:00.427387    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38b6cfe-6e80-4ebd-afd2-fb972fb14f27-maas-api-service-ca\") pod \"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27\" (UID: \"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27\") "
Sep 10 19:47:00.427808 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:00.427786    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f38b6cfe-6e80-4ebd-afd2-fb972fb14f27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f38b6cfe-6e80-4ebd-afd2-fb972fb14f27" (UID: "f38b6cfe-6e80-4ebd-afd2-fb972fb14f27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:47:00.528400 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:00.528362    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f38b6cfe-6e80-4ebd-afd2-fb972fb14f27-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:47:01.179748 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:01.179716    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"
Sep 10 19:47:01.179748 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:01.179741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp" event={"ID":"f38b6cfe-6e80-4ebd-afd2-fb972fb14f27","Type":"ContainerDied","Data":"3faac2b35bcb03a55d99ec22e0025c8bdf512c52aec81da0ce734f8d79790633"}
Sep 10 19:47:01.179977 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:01.179799    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3faac2b35bcb03a55d99ec22e0025c8bdf512c52aec81da0ce734f8d79790633"
Sep 10 19:47:01.216461 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:47:01.216421    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38b6cfe_6e80_4ebd_afd2_fb972fb14f27.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 19:47:08.295500 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.295466    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:08.378206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.378104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:08.467426 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.467386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:08.551828 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.551792    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"]
Sep 10 19:47:08.563913 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.563859    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-ff3019-gs8qp"]
Sep 10 19:47:08.572534 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.572493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:08.670846 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.670768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:08.735125 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.735084    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:47:08.984423 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:08.984329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:09.696069 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:09.696039    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f38b6cfe-6e80-4ebd-afd2-fb972fb14f27" path="/var/lib/kubelet/pods/f38b6cfe-6e80-4ebd-afd2-fb972fb14f27/volumes"
Sep 10 19:47:12.845116 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:12.845065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:12.914880 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:12.914838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:13.257967 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:13.257931    2562 generic.go:356] "Generic (PLEG): container finished" podID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerID="11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8" exitCode=0
Sep 10 19:47:13.258214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:13.258009    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" event={"ID":"11a4a27c-99fb-48cb-9f93-996855f036bd","Type":"ContainerDied","Data":"11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8"}
Sep 10 19:47:13.258485 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:13.258467    2562 scope.go:117] "RemoveContainer" containerID="11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8"
Sep 10 19:47:13.259638 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:13.259607    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:47:13.259704 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:13.259691    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:47:13.259747 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:13.259723    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 19:47:14.271105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:14.271070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" event={"ID":"11a4a27c-99fb-48cb-9f93-996855f036bd","Type":"ContainerStarted","Data":"7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33"}
Sep 10 19:47:14.959795 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:14.959747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:15.061576 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:15.061497    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:15.197407 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:15.197375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:47:15.197683 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:15.197657    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 10 19:47:15.197747 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:15.197728    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 10 19:47:15.231659 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:15.231552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:16.197687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:16.197651    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 10 19:47:16.198110 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:16.197713    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 10 19:47:16.661800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:16.661755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:17.197432 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:17.197395    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 10 19:47:17.197628 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:17.197456    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 10 19:47:18.196957 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:18.196913    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 10 19:47:18.197348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:18.196985    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 10 19:47:18.407292 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:18.407251    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:47:19.162957 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:19.162919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:19.197696 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:19.197658    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 10 19:47:19.198096 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:19.197757    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 10 19:47:20.201075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:20.201038    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:47:20.201478 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:20.201314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:47:20.202072 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:20.202052    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:47:41.256288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:41.256247    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:47:45.903281 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:45.903240    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:47:59.340518 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:47:59.340477    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:48:05.423809 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:05.423764    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8_ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2/payload-processing/0.log"
Sep 10 19:48:05.982575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:05.982539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:06.405675 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:06.405629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:06.697324 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:06.697224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:10.228749 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:10.228698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:10.385420 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:10.385381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:11.648219 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:11.648178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:11.758715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:11.758682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:11.909054 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:11.908956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:12.381856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:12.381759    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:12.614781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:12.614745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:17.246620 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:17.246576    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:48:40.436004 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:40.435915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:41.215588 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.215549    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"]
Sep 10 19:48:41.216027 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.216010    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38b6cfe-6e80-4ebd-afd2-fb972fb14f27" containerName="revoke-keys"
Sep 10 19:48:41.216079 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.216029    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38b6cfe-6e80-4ebd-afd2-fb972fb14f27" containerName="revoke-keys"
Sep 10 19:48:41.216116 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.216105    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38b6cfe-6e80-4ebd-afd2-fb972fb14f27" containerName="revoke-keys"
Sep 10 19:48:41.219288 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.219266    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"
Sep 10 19:48:41.222124 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.222098    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:48:41.222252 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.222161    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:48:41.229016 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.228971    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"]
Sep 10 19:48:41.319762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.319718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f3e70b-79be-4d81-9b6f-d59b107bf2b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj\" (UID: \"89f3e70b-79be-4d81-9b6f-d59b107bf2b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"
Sep 10 19:48:41.420278 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.420237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f3e70b-79be-4d81-9b6f-d59b107bf2b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj\" (UID: \"89f3e70b-79be-4d81-9b6f-d59b107bf2b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"
Sep 10 19:48:41.420933 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.420912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f3e70b-79be-4d81-9b6f-d59b107bf2b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj\" (UID: \"89f3e70b-79be-4d81-9b6f-d59b107bf2b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"
Sep 10 19:48:41.531721 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.531616    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"
Sep 10 19:48:41.827090 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:41.827061    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"]
Sep 10 19:48:41.829929 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:48:41.829902    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f3e70b_79be_4d81_9b6f_d59b107bf2b3.slice/crio-7f3386d6d760a504dee7071e7ff3b13500a437da0f1b2e264cc142042801eac1 WatchSource:0}: Error finding container 7f3386d6d760a504dee7071e7ff3b13500a437da0f1b2e264cc142042801eac1: Status 404 returned error can't find the container with id 7f3386d6d760a504dee7071e7ff3b13500a437da0f1b2e264cc142042801eac1
Sep 10 19:48:42.767509 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:42.767471    2562 generic.go:356] "Generic (PLEG): container finished" podID="89f3e70b-79be-4d81-9b6f-d59b107bf2b3" containerID="7e87befb8fae3838b82eb168efce9efeea0d6722dd9b0420b76ead109df236ef" exitCode=0
Sep 10 19:48:42.767942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:42.767552    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj" event={"ID":"89f3e70b-79be-4d81-9b6f-d59b107bf2b3","Type":"ContainerDied","Data":"7e87befb8fae3838b82eb168efce9efeea0d6722dd9b0420b76ead109df236ef"}
Sep 10 19:48:42.767942 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:42.767606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj" event={"ID":"89f3e70b-79be-4d81-9b6f-d59b107bf2b3","Type":"ContainerStarted","Data":"7f3386d6d760a504dee7071e7ff3b13500a437da0f1b2e264cc142042801eac1"}
Sep 10 19:48:43.915356 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:43.915320    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"
Sep 10 19:48:43.944790 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:43.944751    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f3e70b-79be-4d81-9b6f-d59b107bf2b3-maas-api-service-ca\") pod \"89f3e70b-79be-4d81-9b6f-d59b107bf2b3\" (UID: \"89f3e70b-79be-4d81-9b6f-d59b107bf2b3\") "
Sep 10 19:48:43.945160 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:43.945133    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89f3e70b-79be-4d81-9b6f-d59b107bf2b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89f3e70b-79be-4d81-9b6f-d59b107bf2b3" (UID: "89f3e70b-79be-4d81-9b6f-d59b107bf2b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:48:43.945269 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:43.945232    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f3e70b-79be-4d81-9b6f-d59b107bf2b3-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:48:44.783647 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:44.783614    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj" event={"ID":"89f3e70b-79be-4d81-9b6f-d59b107bf2b3","Type":"ContainerDied","Data":"7f3386d6d760a504dee7071e7ff3b13500a437da0f1b2e264cc142042801eac1"}
Sep 10 19:48:44.783647 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:44.783632    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"
Sep 10 19:48:44.783864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:44.783663    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f3386d6d760a504dee7071e7ff3b13500a437da0f1b2e264cc142042801eac1"
Sep 10 19:48:52.511840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:52.511802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:52.612990 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:52.612945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:52.712713 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:52.712668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:52.813756 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:52.813648    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"]
Sep 10 19:48:52.816113 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:52.816086    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-855446-nxjnj"]
Sep 10 19:48:52.896648 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:52.896608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:53.373557 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:53.373507    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:53.695742 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:53.695706    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89f3e70b-79be-4d81-9b6f-d59b107bf2b3" path="/var/lib/kubelet/pods/89f3e70b-79be-4d81-9b6f-d59b107bf2b3/volumes"
Sep 10 19:48:57.004940 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:57.004897    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:48:59.326123 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:59.326087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:59.428973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:59.428936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:48:59.614474 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:48:59.614375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:03.561562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:03.561527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:08.114822 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:08.114787    2562 scope.go:117] "RemoveContainer" containerID="e9b64736c247d59a6508e76a38e0c4477ebe440b7608a56f70ede1685852e2db"
Sep 10 19:49:16.725191 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:16.725148    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:49:19.062224 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.062175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:19.123710 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.123675    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"]
Sep 10 19:49:19.123970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.123941    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8" podUID="ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" containerName="payload-processing" containerID="cri-o://2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8" gracePeriod=30
Sep 10 19:49:19.144219 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.144193    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"]
Sep 10 19:49:19.144435 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.144413    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf" podUID="9081bfa1-158d-4993-8319-24bc4991513b" containerName="payload-pre-processing" containerID="cri-o://36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18" gracePeriod=30
Sep 10 19:49:19.228768 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.228707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:19.356683 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.356586    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:19.426840 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.426812    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:49:19.430105 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.430082    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:49:19.469518 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.469459    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:19.489257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.489223    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmmbn\" (UniqueName: \"kubernetes.io/projected/9081bfa1-158d-4993-8319-24bc4991513b-kube-api-access-qmmbn\") pod \"9081bfa1-158d-4993-8319-24bc4991513b\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") "
Sep 10 19:49:19.489257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.489263    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhng4\" (UniqueName: \"kubernetes.io/projected/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-kube-api-access-fhng4\") pod \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") "
Sep 10 19:49:19.489522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.489291    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-plugins-config-volume\") pod \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\" (UID: \"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2\") "
Sep 10 19:49:19.489522 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.489323    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9081bfa1-158d-4993-8319-24bc4991513b-plugins-config-volume\") pod \"9081bfa1-158d-4993-8319-24bc4991513b\" (UID: \"9081bfa1-158d-4993-8319-24bc4991513b\") "
Sep 10 19:49:19.489724 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.489683    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" (UID: "ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:49:19.489837 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.489744    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9081bfa1-158d-4993-8319-24bc4991513b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9081bfa1-158d-4993-8319-24bc4991513b" (UID: "9081bfa1-158d-4993-8319-24bc4991513b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:49:19.491528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.491493    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9081bfa1-158d-4993-8319-24bc4991513b-kube-api-access-qmmbn" (OuterVolumeSpecName: "kube-api-access-qmmbn") pod "9081bfa1-158d-4993-8319-24bc4991513b" (UID: "9081bfa1-158d-4993-8319-24bc4991513b"). InnerVolumeSpecName "kube-api-access-qmmbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:49:19.491639 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.491529    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-kube-api-access-fhng4" (OuterVolumeSpecName: "kube-api-access-fhng4") pod "ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" (UID: "ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2"). InnerVolumeSpecName "kube-api-access-fhng4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:49:19.577064 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.577017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:19.590335 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.590290    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9081bfa1-158d-4993-8319-24bc4991513b-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:49:19.590335 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.590328    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmmbn\" (UniqueName: \"kubernetes.io/projected/9081bfa1-158d-4993-8319-24bc4991513b-kube-api-access-qmmbn\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:49:19.590335 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.590342    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhng4\" (UniqueName: \"kubernetes.io/projected/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-kube-api-access-fhng4\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:49:19.590550 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.590357    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:49:19.882937 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.882881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:19.983575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.983470    2562 generic.go:356] "Generic (PLEG): container finished" podID="ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" containerID="2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8" exitCode=0
Sep 10 19:49:19.983575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.983511    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"
Sep 10 19:49:19.983575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.983561    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8" event={"ID":"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2","Type":"ContainerDied","Data":"2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8"}
Sep 10 19:49:19.983864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.983621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8" event={"ID":"ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2","Type":"ContainerDied","Data":"c282c23270c8864e8a07072a0c76b462bbc9f2f758b52671cac3a4731e1bdc20"}
Sep 10 19:49:19.983864 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.983658    2562 scope.go:117] "RemoveContainer" containerID="2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8"
Sep 10 19:49:19.985047 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.985027    2562 generic.go:356] "Generic (PLEG): container finished" podID="9081bfa1-158d-4993-8319-24bc4991513b" containerID="36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18" exitCode=0
Sep 10 19:49:19.985132 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.985074    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"
Sep 10 19:49:19.985132 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.985111    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf" event={"ID":"9081bfa1-158d-4993-8319-24bc4991513b","Type":"ContainerDied","Data":"36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18"}
Sep 10 19:49:19.985214 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.985155    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf" event={"ID":"9081bfa1-158d-4993-8319-24bc4991513b","Type":"ContainerDied","Data":"88dbeaa4e9d3d19ce7eaf1a73d60054d2625dd76699e571b7b3b6e0fbf299b82"}
Sep 10 19:49:19.993325 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.993307    2562 scope.go:117] "RemoveContainer" containerID="2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8"
Sep 10 19:49:19.993657 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:49:19.993636    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8\": container with ID starting with 2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8 not found: ID does not exist" containerID="2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8"
Sep 10 19:49:19.993743 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.993664    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8"} err="failed to get container status \"2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8\": rpc error: code = NotFound desc = could not find container \"2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8\": container with ID starting with 2d99230493ab3c420ed926b6339047c44f48d3b3153fa20065b294ca539aacd8 not found: ID does not exist"
Sep 10 19:49:19.993743 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:19.993682    2562 scope.go:117] "RemoveContainer" containerID="36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18"
Sep 10 19:49:20.002501 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:20.002478    2562 scope.go:117] "RemoveContainer" containerID="36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18"
Sep 10 19:49:20.002778 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:49:20.002761    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18\": container with ID starting with 36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18 not found: ID does not exist" containerID="36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18"
Sep 10 19:49:20.002832 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:20.002784    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18"} err="failed to get container status \"36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18\": rpc error: code = NotFound desc = could not find container \"36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18\": container with ID starting with 36b88d1ff1f495ab640ee8d8dfa19443b7a6886a77dcdd2ff740df68d6f5da18 not found: ID does not exist"
Sep 10 19:49:20.009532 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:20.009504    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"]
Sep 10 19:49:20.014734 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:20.014707    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-836291-86b9754bbc-cjrx8"]
Sep 10 19:49:20.027755 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:20.027727    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"]
Sep 10 19:49:20.033012 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:20.032986    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-836291-985f58d74-96pjf"]
Sep 10 19:49:21.696772 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:21.696739    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9081bfa1-158d-4993-8319-24bc4991513b" path="/var/lib/kubelet/pods/9081bfa1-158d-4993-8319-24bc4991513b/volumes"
Sep 10 19:49:21.697158 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:21.697143    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" path="/var/lib/kubelet/pods/ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2/volumes"
Sep 10 19:49:24.940413 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:24.940351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:29.721742 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:29.721704    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:49:33.546520 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:33.546476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:49:38.916207 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:49:38.916157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:00.737586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:00.737492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:01.462971 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:01.462929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:02.300306 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:02.300268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:07.694448 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:07.694416    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:50:07.699279 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:07.699256    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:50:07.851766 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:07.851738    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:50:07.856378 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:07.856352    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:50:08.157213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:08.157177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:22.850190 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:22.850144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:22.987650 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:22.987609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:23.196774 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:23.196733    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"]
Sep 10 19:50:23.197053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:23.197024    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy" containerID="cri-o://5555a3c0da59f031e9cc61115ec4f2c0027f2781de5c6441a1f8ceea6c3806bb" gracePeriod=30
Sep 10 19:50:23.249753 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:23.249710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:23.686748 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:23.686705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:28.369218 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.368935    2562 generic.go:356] "Generic (PLEG): container finished" podID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerID="5555a3c0da59f031e9cc61115ec4f2c0027f2781de5c6441a1f8ceea6c3806bb" exitCode=0
Sep 10 19:50:28.369218 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.369020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" event={"ID":"a0c4a233-f452-477d-87ee-6fef79b90dc9","Type":"ContainerDied","Data":"5555a3c0da59f031e9cc61115ec4f2c0027f2781de5c6441a1f8ceea6c3806bb"}
Sep 10 19:50:28.369218 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.369086    2562 scope.go:117] "RemoveContainer" containerID="07b8b3ed5a31c9fda8b291f5c882086aeff79c80441f5c643d01fbdd5bb23f9d"
Sep 10 19:50:28.459329 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.459255    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:50:28.556726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.556674    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c4a233-f452-477d-87ee-6fef79b90dc9-istiod-ca-cert\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.556726 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.556726    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-envoy\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557000 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.556758    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-socket\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557000 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.556875    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-certs\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557000 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.556978    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-credential-socket\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557025    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-data\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-token\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557082    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:28.557159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557133    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-podinfo\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557178    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfszb\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-kube-api-access-nfszb\") pod \"a0c4a233-f452-477d-87ee-6fef79b90dc9\" (UID: \"a0c4a233-f452-477d-87ee-6fef79b90dc9\") "
Sep 10 19:50:28.557360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0c4a233-f452-477d-87ee-6fef79b90dc9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:50:28.557360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557245    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:28.557360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557287    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:28.557360 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557308    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:28.557602 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557570    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-credential-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.557602 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557588    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-data\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.557701 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557603    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c4a233-f452-477d-87ee-6fef79b90dc9-istiod-ca-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.557701 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557617    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.557701 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.557631    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-workload-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.559181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.559145    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:28.559314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.559204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:50:28.559656 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.559632    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-kube-api-access-nfszb" (OuterVolumeSpecName: "kube-api-access-nfszb") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "kube-api-access-nfszb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:28.559743 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.559657    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0c4a233-f452-477d-87ee-6fef79b90dc9" (UID: "a0c4a233-f452-477d-87ee-6fef79b90dc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:28.659143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.659101    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-envoy\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.659143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.659134    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-token\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.659143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.659143    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c4a233-f452-477d-87ee-6fef79b90dc9-istio-podinfo\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:28.659143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:28.659154    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfszb\" (UniqueName: \"kubernetes.io/projected/a0c4a233-f452-477d-87ee-6fef79b90dc9-kube-api-access-nfszb\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:29.375878 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:29.375849    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"
Sep 10 19:50:29.376295 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:29.375882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g" event={"ID":"a0c4a233-f452-477d-87ee-6fef79b90dc9","Type":"ContainerDied","Data":"46fb4186d75d9645d6fd622f9b35e49a14a5ed96215e0366eab33d9210b545de"}
Sep 10 19:50:29.376295 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:29.375987    2562 scope.go:117] "RemoveContainer" containerID="5555a3c0da59f031e9cc61115ec4f2c0027f2781de5c6441a1f8ceea6c3806bb"
Sep 10 19:50:29.402402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:29.402366    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"]
Sep 10 19:50:29.407800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:29.407771    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-836291-openshift-default-6c4bb7566d-nhv4g"]
Sep 10 19:50:29.695781 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:29.695742    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" path="/var/lib/kubelet/pods/a0c4a233-f452-477d-87ee-6fef79b90dc9/volumes"
Sep 10 19:50:34.167264 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:34.167219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:34.243645 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:34.243598    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:34.315800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:34.315762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:34.404332 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:34.404299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:34.479786 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:34.479688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:34.910571 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:34.910522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:37.308460 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:37.308419    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:50:41.243695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:41.243660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:41.416307 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:41.416266    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:41.431866 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:41.431829    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"]
Sep 10 19:50:41.432095 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:41.432066    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy" containerID="cri-o://7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33" gracePeriod=30
Sep 10 19:50:41.691429 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:41.691388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:42.038346 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:42.038249    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:46.695041 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.694971    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:50:46.733314 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-envoy\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733361    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-data\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a4a27c-99fb-48cb-9f93-996855f036bd-istiod-ca-cert\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733410    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bcpj\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-kube-api-access-8bcpj\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733435    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-podinfo\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733463    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-certs\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733501    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-credential-socket\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733528    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-socket\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733655    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-token\") pod \"11a4a27c-99fb-48cb-9f93-996855f036bd\" (UID: \"11a4a27c-99fb-48cb-9f93-996855f036bd\") "
Sep 10 19:50:46.733830 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733684    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:46.734025 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733907    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:46.734025 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733949    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a4a27c-99fb-48cb-9f93-996855f036bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:50:46.734136 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.733952    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:46.734136 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.734110    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-data\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.734136 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.734131    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11a4a27c-99fb-48cb-9f93-996855f036bd-istiod-ca-cert\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.734294 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.734144    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.734294 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.734233    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:46.735932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.735877    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-kube-api-access-8bcpj" (OuterVolumeSpecName: "kube-api-access-8bcpj") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "kube-api-access-8bcpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:46.736120 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.736099    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:46.736120 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.736110    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:50:46.736280 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.736263    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11a4a27c-99fb-48cb-9f93-996855f036bd" (UID: "11a4a27c-99fb-48cb-9f93-996855f036bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:46.834800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.834758    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bcpj\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-kube-api-access-8bcpj\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.834800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.834794    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-podinfo\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.834800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.834803    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-workload-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.834800 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.834813    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-credential-socket\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.835106 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.834822    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-token\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:46.835106 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:46.834829    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11a4a27c-99fb-48cb-9f93-996855f036bd-istio-envoy\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:47.480869 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.480835    2562 generic.go:356] "Generic (PLEG): container finished" podID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerID="7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33" exitCode=0
Sep 10 19:50:47.481071 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.480882    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"
Sep 10 19:50:47.481071 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.480931    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" event={"ID":"11a4a27c-99fb-48cb-9f93-996855f036bd","Type":"ContainerDied","Data":"7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33"}
Sep 10 19:50:47.481071 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.480996    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc" event={"ID":"11a4a27c-99fb-48cb-9f93-996855f036bd","Type":"ContainerDied","Data":"e2921b21f0779f246a44d80556971d019d6a54e10b72faa802d443d643003b21"}
Sep 10 19:50:47.481071 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.481026    2562 scope.go:117] "RemoveContainer" containerID="7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33"
Sep 10 19:50:47.496399 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.492880    2562 scope.go:117] "RemoveContainer" containerID="11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8"
Sep 10 19:50:47.502502 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.502481    2562 scope.go:117] "RemoveContainer" containerID="7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33"
Sep 10 19:50:47.502847 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:50:47.502826    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33\": container with ID starting with 7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33 not found: ID does not exist" containerID="7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33"
Sep 10 19:50:47.502940 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.502861    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33"} err="failed to get container status \"7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33\": rpc error: code = NotFound desc = could not find container \"7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33\": container with ID starting with 7399ecffba2ab89939d5d1b2e24249f80ee95a6f0793720dc3e204a0ea3acd33 not found: ID does not exist"
Sep 10 19:50:47.502940 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.502900    2562 scope.go:117] "RemoveContainer" containerID="11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8"
Sep 10 19:50:47.503210 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:50:47.503181    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8\": container with ID starting with 11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8 not found: ID does not exist" containerID="11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8"
Sep 10 19:50:47.503251 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.503217    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8"} err="failed to get container status \"11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8\": rpc error: code = NotFound desc = could not find container \"11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8\": container with ID starting with 11558c1b6694a1e7753b4a33a8c6ce34ab663c73e77694d5bbc2080b5b6ac7f8 not found: ID does not exist"
Sep 10 19:50:47.512361 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.512331    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"]
Sep 10 19:50:47.516246 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.516214    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-b588eb-openshift-default-7d4cf9cbf6-zr4kc"]
Sep 10 19:50:47.695535 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.695491    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" path="/var/lib/kubelet/pods/11a4a27c-99fb-48cb-9f93-996855f036bd/volumes"
Sep 10 19:50:47.967934 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:47.967864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:52.338419 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.338369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:52.339798 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.339771    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"]
Sep 10 19:50:52.344997 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.344971    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-07c5fe-29817825-bqlz6"]
Sep 10 19:50:52.372526 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.372491    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"]
Sep 10 19:50:52.372773 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.372747    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx" podUID="5cb8fc27-d99f-4a49-9f47-78999ac5becc" containerName="payload-processing" containerID="cri-o://906530d57edfc19478535c45fb4643b65cc48defe84956fc449490d0a95c918e" gracePeriod=30
Sep 10 19:50:52.381703 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.381663    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"]
Sep 10 19:50:52.381975 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.381941    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq" podUID="0a534b13-f63b-4512-8283-a10d7c324f9f" containerName="payload-pre-processing" containerID="cri-o://6c403310b303a94234faa7e3b46d8cd2f55fcfd270d29f86abbf7844a882ac1a" gracePeriod=30
Sep 10 19:50:52.438643 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.438609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:52.469462 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.469417    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx" podUID="5cb8fc27-d99f-4a49-9f47-78999ac5becc" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.64:9004: connect: connection refused"
Sep 10 19:50:52.469663 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.469417    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq" podUID="0a534b13-f63b-4512-8283-a10d7c324f9f" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.65:9004: connect: connection refused"
Sep 10 19:50:52.513154 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.513117    2562 generic.go:356] "Generic (PLEG): container finished" podID="0a534b13-f63b-4512-8283-a10d7c324f9f" containerID="6c403310b303a94234faa7e3b46d8cd2f55fcfd270d29f86abbf7844a882ac1a" exitCode=0
Sep 10 19:50:52.513318 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.513197    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq" event={"ID":"0a534b13-f63b-4512-8283-a10d7c324f9f","Type":"ContainerDied","Data":"6c403310b303a94234faa7e3b46d8cd2f55fcfd270d29f86abbf7844a882ac1a"}
Sep 10 19:50:52.515335 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.515299    2562 generic.go:356] "Generic (PLEG): container finished" podID="5cb8fc27-d99f-4a49-9f47-78999ac5becc" containerID="906530d57edfc19478535c45fb4643b65cc48defe84956fc449490d0a95c918e" exitCode=0
Sep 10 19:50:52.515536 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.515366    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx" event={"ID":"5cb8fc27-d99f-4a49-9f47-78999ac5becc","Type":"ContainerDied","Data":"906530d57edfc19478535c45fb4643b65cc48defe84956fc449490d0a95c918e"}
Sep 10 19:50:52.537409 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.537364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:52.637974 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.637937    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:52.660562 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.660539    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:50:52.663666 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.663638    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:50:52.789838 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.789797    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a534b13-f63b-4512-8283-a10d7c324f9f-plugins-config-volume\") pod \"0a534b13-f63b-4512-8283-a10d7c324f9f\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") "
Sep 10 19:50:52.790046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.789878    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwdgb\" (UniqueName: \"kubernetes.io/projected/0a534b13-f63b-4512-8283-a10d7c324f9f-kube-api-access-vwdgb\") pod \"0a534b13-f63b-4512-8283-a10d7c324f9f\" (UID: \"0a534b13-f63b-4512-8283-a10d7c324f9f\") "
Sep 10 19:50:52.790046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.789931    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z92d\" (UniqueName: \"kubernetes.io/projected/5cb8fc27-d99f-4a49-9f47-78999ac5becc-kube-api-access-7z92d\") pod \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") "
Sep 10 19:50:52.790046 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.790014    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb8fc27-d99f-4a49-9f47-78999ac5becc-plugins-config-volume\") pod \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\" (UID: \"5cb8fc27-d99f-4a49-9f47-78999ac5becc\") "
Sep 10 19:50:52.790347 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.790304    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a534b13-f63b-4512-8283-a10d7c324f9f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a534b13-f63b-4512-8283-a10d7c324f9f" (UID: "0a534b13-f63b-4512-8283-a10d7c324f9f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:50:52.790420 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.790397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb8fc27-d99f-4a49-9f47-78999ac5becc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5cb8fc27-d99f-4a49-9f47-78999ac5becc" (UID: "5cb8fc27-d99f-4a49-9f47-78999ac5becc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:50:52.792170 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.792151    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a534b13-f63b-4512-8283-a10d7c324f9f-kube-api-access-vwdgb" (OuterVolumeSpecName: "kube-api-access-vwdgb") pod "0a534b13-f63b-4512-8283-a10d7c324f9f" (UID: "0a534b13-f63b-4512-8283-a10d7c324f9f"). InnerVolumeSpecName "kube-api-access-vwdgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:52.792213 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.792178    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb8fc27-d99f-4a49-9f47-78999ac5becc-kube-api-access-7z92d" (OuterVolumeSpecName: "kube-api-access-7z92d") pod "5cb8fc27-d99f-4a49-9f47-78999ac5becc" (UID: "5cb8fc27-d99f-4a49-9f47-78999ac5becc"). InnerVolumeSpecName "kube-api-access-7z92d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:52.891273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.891239    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb8fc27-d99f-4a49-9f47-78999ac5becc-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:52.891273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.891268    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a534b13-f63b-4512-8283-a10d7c324f9f-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:52.891273 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.891280    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwdgb\" (UniqueName: \"kubernetes.io/projected/0a534b13-f63b-4512-8283-a10d7c324f9f-kube-api-access-vwdgb\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:52.891508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:52.891289    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z92d\" (UniqueName: \"kubernetes.io/projected/5cb8fc27-d99f-4a49-9f47-78999ac5becc-kube-api-access-7z92d\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:50:53.006351 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.006312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:53.521957 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.521927    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"
Sep 10 19:50:53.522390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.521955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx" event={"ID":"5cb8fc27-d99f-4a49-9f47-78999ac5becc","Type":"ContainerDied","Data":"34f4071c3c0f0cb7cb359f9e3d0e599d4a68ae6b6bf470b4771c0e70845d1f0f"}
Sep 10 19:50:53.522390 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.522017    2562 scope.go:117] "RemoveContainer" containerID="906530d57edfc19478535c45fb4643b65cc48defe84956fc449490d0a95c918e"
Sep 10 19:50:53.532337 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.532217    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq" event={"ID":"0a534b13-f63b-4512-8283-a10d7c324f9f","Type":"ContainerDied","Data":"c1cc34f430e88e3a211a56f8e1f2aa703a5333be474f61c278e8567f35c69469"}
Sep 10 19:50:53.532437 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.532368    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"
Sep 10 19:50:53.540604 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.540575    2562 scope.go:117] "RemoveContainer" containerID="6c403310b303a94234faa7e3b46d8cd2f55fcfd270d29f86abbf7844a882ac1a"
Sep 10 19:50:53.549625 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.549603    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"]
Sep 10 19:50:53.554518 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.554486    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-07c5fe-5d5cb8d4c5-s9jvx"]
Sep 10 19:50:53.568665 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.568631    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"]
Sep 10 19:50:53.575443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.575402    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-07c5fe-d8db6764b-98hpq"]
Sep 10 19:50:53.696866 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.696782    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a534b13-f63b-4512-8283-a10d7c324f9f" path="/var/lib/kubelet/pods/0a534b13-f63b-4512-8283-a10d7c324f9f/volumes"
Sep 10 19:50:53.697193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.697179    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb8fc27-d99f-4a49-9f47-78999ac5becc" path="/var/lib/kubelet/pods/5cb8fc27-d99f-4a49-9f47-78999ac5becc/volumes"
Sep 10 19:50:53.697426 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:53.697415    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc1d93e0-a744-473c-88e7-f7bab356755e" path="/var/lib/kubelet/pods/fc1d93e0-a744-473c-88e7-f7bab356755e/volumes"
Sep 10 19:50:55.303928 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:55.303875    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:50:59.420442 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:59.420407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:59.523980 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:59.523936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:50:59.734831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:50:59.734705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:00.043209 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043123    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"]
Sep 10 19:51:00.043530 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043518    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9081bfa1-158d-4993-8319-24bc4991513b" containerName="payload-pre-processing"
Sep 10 19:51:00.043575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043531    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9081bfa1-158d-4993-8319-24bc4991513b" containerName="payload-pre-processing"
Sep 10 19:51:00.043575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043548    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy"
Sep 10 19:51:00.043575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043554    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy"
Sep 10 19:51:00.043575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043561    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" containerName="payload-processing"
Sep 10 19:51:00.043575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043567    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" containerName="payload-processing"
Sep 10 19:51:00.043575 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043576    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a534b13-f63b-4512-8283-a10d7c324f9f" containerName="payload-pre-processing"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043582    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a534b13-f63b-4512-8283-a10d7c324f9f" containerName="payload-pre-processing"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043590    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb8fc27-d99f-4a49-9f47-78999ac5becc" containerName="payload-processing"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043596    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb8fc27-d99f-4a49-9f47-78999ac5becc" containerName="payload-processing"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043602    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89f3e70b-79be-4d81-9b6f-d59b107bf2b3" containerName="revoke-keys"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043607    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="89f3e70b-79be-4d81-9b6f-d59b107bf2b3" containerName="revoke-keys"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043619    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043625    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043633    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043639    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043696    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043703    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a534b13-f63b-4512-8283-a10d7c324f9f" containerName="payload-pre-processing"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043709    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9081bfa1-158d-4993-8319-24bc4991513b" containerName="payload-pre-processing"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043715    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043721    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043727    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb8fc27-d99f-4a49-9f47-78999ac5becc" containerName="payload-processing"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043734    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c4a233-f452-477d-87ee-6fef79b90dc9" containerName="istio-proxy"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043739    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="89f3e70b-79be-4d81-9b6f-d59b107bf2b3" containerName="revoke-keys"
Sep 10 19:51:00.043762 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.043745    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac23ee40-e7ea-4d50-aaf2-2e2193c9bfa2" containerName="payload-processing"
Sep 10 19:51:00.048215 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.048193    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"
Sep 10 19:51:00.051053 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.051024    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:51:00.051187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.051024    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zztzd\""
Sep 10 19:51:00.055987 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.055759    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"]
Sep 10 19:51:00.157086 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.157046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b22179c-1ee1-40f3-94ed-9d14479f29ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99\" (UID: \"4b22179c-1ee1-40f3-94ed-9d14479f29ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"
Sep 10 19:51:00.258581 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.258534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b22179c-1ee1-40f3-94ed-9d14479f29ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99\" (UID: \"4b22179c-1ee1-40f3-94ed-9d14479f29ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"
Sep 10 19:51:00.259257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.259236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b22179c-1ee1-40f3-94ed-9d14479f29ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99\" (UID: \"4b22179c-1ee1-40f3-94ed-9d14479f29ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"
Sep 10 19:51:00.308187 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.308094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:00.361008 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.360974    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"
Sep 10 19:51:00.638163 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:00.638132    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"]
Sep 10 19:51:00.640933 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:51:00.640902    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b22179c_1ee1_40f3_94ed_9d14479f29ff.slice/crio-5f96a709d74e50c1fe403ac9bb426db4ea5d28126897fbbbd3d9108358df56b5 WatchSource:0}: Error finding container 5f96a709d74e50c1fe403ac9bb426db4ea5d28126897fbbbd3d9108358df56b5: Status 404 returned error can't find the container with id 5f96a709d74e50c1fe403ac9bb426db4ea5d28126897fbbbd3d9108358df56b5
Sep 10 19:51:01.577148 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:01.577113    2562 generic.go:356] "Generic (PLEG): container finished" podID="4b22179c-1ee1-40f3-94ed-9d14479f29ff" containerID="55070e3ad8ddeef31c9952342d8a451aaf772618a3c9e3cb2a61c15bae6dc958" exitCode=0
Sep 10 19:51:01.577334 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:01.577191    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99" event={"ID":"4b22179c-1ee1-40f3-94ed-9d14479f29ff","Type":"ContainerDied","Data":"55070e3ad8ddeef31c9952342d8a451aaf772618a3c9e3cb2a61c15bae6dc958"}
Sep 10 19:51:01.577334 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:01.577244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99" event={"ID":"4b22179c-1ee1-40f3-94ed-9d14479f29ff","Type":"ContainerStarted","Data":"5f96a709d74e50c1fe403ac9bb426db4ea5d28126897fbbbd3d9108358df56b5"}
Sep 10 19:51:02.714453 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:02.714425    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"
Sep 10 19:51:02.783284 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:02.783244    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b22179c-1ee1-40f3-94ed-9d14479f29ff-maas-api-service-ca\") pod \"4b22179c-1ee1-40f3-94ed-9d14479f29ff\" (UID: \"4b22179c-1ee1-40f3-94ed-9d14479f29ff\") "
Sep 10 19:51:02.783689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:02.783658    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b22179c-1ee1-40f3-94ed-9d14479f29ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b22179c-1ee1-40f3-94ed-9d14479f29ff" (UID: "4b22179c-1ee1-40f3-94ed-9d14479f29ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:51:02.884302 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:02.884214    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b22179c-1ee1-40f3-94ed-9d14479f29ff-maas-api-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:51:03.589267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:03.589232    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"
Sep 10 19:51:03.589267 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:03.589254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99" event={"ID":"4b22179c-1ee1-40f3-94ed-9d14479f29ff","Type":"ContainerDied","Data":"5f96a709d74e50c1fe403ac9bb426db4ea5d28126897fbbbd3d9108358df56b5"}
Sep 10 19:51:03.589481 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:03.589306    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f96a709d74e50c1fe403ac9bb426db4ea5d28126897fbbbd3d9108358df56b5"
Sep 10 19:51:08.161482 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:08.161440    2562 scope.go:117] "RemoveContainer" containerID="fed444edee309ca6680acb2678a2618a11be7aabb68689bc391952b3a9c3e01f"
Sep 10 19:51:08.170181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:08.170157    2562 scope.go:117] "RemoveContainer" containerID="47af9f460a78160c999fd7c020707ec5586ce5609fb2dd565d5781d5b70c7c11"
Sep 10 19:51:10.692286 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.692252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:10.719517 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.719477    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"]
Sep 10 19:51:10.719825 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.719793    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq" podUID="d79696b1-fe74-4685-a4aa-40ab12471d90" containerName="payload-processing" containerID="cri-o://7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060" gracePeriod=30
Sep 10 19:51:10.787006 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.786972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:10.889717 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.889680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:10.915658 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.915620    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"]
Sep 10 19:51:10.922206 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.922162    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-2a1fa9-srx99"]
Sep 10 19:51:10.979850 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.979823    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:51:10.986715 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:10.986685    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:11.162513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.162464    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79696b1-fe74-4685-a4aa-40ab12471d90-plugins-config-volume\") pod \"d79696b1-fe74-4685-a4aa-40ab12471d90\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") "
Sep 10 19:51:11.162731 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.162561    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzm5s\" (UniqueName: \"kubernetes.io/projected/d79696b1-fe74-4685-a4aa-40ab12471d90-kube-api-access-kzm5s\") pod \"d79696b1-fe74-4685-a4aa-40ab12471d90\" (UID: \"d79696b1-fe74-4685-a4aa-40ab12471d90\") "
Sep 10 19:51:11.162968 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.162941    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d79696b1-fe74-4685-a4aa-40ab12471d90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d79696b1-fe74-4685-a4aa-40ab12471d90" (UID: "d79696b1-fe74-4685-a4aa-40ab12471d90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:51:11.164734 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.164706    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d79696b1-fe74-4685-a4aa-40ab12471d90-kube-api-access-kzm5s" (OuterVolumeSpecName: "kube-api-access-kzm5s") pod "d79696b1-fe74-4685-a4aa-40ab12471d90" (UID: "d79696b1-fe74-4685-a4aa-40ab12471d90"). InnerVolumeSpecName "kube-api-access-kzm5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:51:11.263959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.263827    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d79696b1-fe74-4685-a4aa-40ab12471d90-plugins-config-volume\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:51:11.263959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.263872    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzm5s\" (UniqueName: \"kubernetes.io/projected/d79696b1-fe74-4685-a4aa-40ab12471d90-kube-api-access-kzm5s\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:51:11.325708 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.325660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:11.634056 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.633968    2562 generic.go:356] "Generic (PLEG): container finished" podID="d79696b1-fe74-4685-a4aa-40ab12471d90" containerID="7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060" exitCode=0
Sep 10 19:51:11.634056 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.634017    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"
Sep 10 19:51:11.634240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.634060    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq" event={"ID":"d79696b1-fe74-4685-a4aa-40ab12471d90","Type":"ContainerDied","Data":"7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060"}
Sep 10 19:51:11.634240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.634114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq" event={"ID":"d79696b1-fe74-4685-a4aa-40ab12471d90","Type":"ContainerDied","Data":"7898417fa0fbb0d52072b63e7af17883ab7c3bad50e329e0bff0c433c8a3474e"}
Sep 10 19:51:11.634240 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.634142    2562 scope.go:117] "RemoveContainer" containerID="7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060"
Sep 10 19:51:11.646935 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.646911    2562 scope.go:117] "RemoveContainer" containerID="7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060"
Sep 10 19:51:11.647277 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:51:11.647254    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060\": container with ID starting with 7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060 not found: ID does not exist" containerID="7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060"
Sep 10 19:51:11.647348 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.647286    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060"} err="failed to get container status \"7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060\": rpc error: code = NotFound desc = could not find container \"7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060\": container with ID starting with 7474712df56302477e54173549f5dc1f8586938c7fe21b4a962295b4f5faf060 not found: ID does not exist"
Sep 10 19:51:11.669637 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.669602    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"]
Sep 10 19:51:11.682848 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.682817    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-2a1fa9-5f66675f7c-62fxq"]
Sep 10 19:51:11.695586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.695558    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b22179c-1ee1-40f3-94ed-9d14479f29ff" path="/var/lib/kubelet/pods/4b22179c-1ee1-40f3-94ed-9d14479f29ff/volumes"
Sep 10 19:51:11.695991 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:11.695975    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d79696b1-fe74-4685-a4aa-40ab12471d90" path="/var/lib/kubelet/pods/d79696b1-fe74-4685-a4aa-40ab12471d90/volumes"
Sep 10 19:51:13.312567 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:13.312518    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:51:17.618085 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:17.618039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:17.715415 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:17.715377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:17.897829 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:17.897794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:51:23.624971 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:51:23.624934    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:52:08.210467 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:08.210433    2562 scope.go:117] "RemoveContainer" containerID="ff1b75852d75749360a86ee9dd64e7ae556b736998725ed84e79bafc531c0bd3"
Sep 10 19:52:08.219079 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:08.219051    2562 scope.go:117] "RemoveContainer" containerID="06f81950e5bf0bd32ddb7f02e9ec603185641cbc7115e44315133af98e37e745"
Sep 10 19:52:24.062442 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:24.062388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:52:36.001495 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:36.001453    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:52:44.258326 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:44.258281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:52:44.636005 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:44.635907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:52:45.307014 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:45.306974    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:52:46.524388 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:52:46.524349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:53:08.246521 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:08.246440    2562 scope.go:117] "RemoveContainer" containerID="49539beb4afc5b8fcaf372bae6f6581573a406ad9fdb32358cea0e961ca44fb1"
Sep 10 19:53:08.254968 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:08.254940    2562 scope.go:117] "RemoveContainer" containerID="282eed2f47240ffc6296fb4b2f9dcbbac660bdf88882f95c07983ad512cca235"
Sep 10 19:53:08.265983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:08.265957    2562 scope.go:117] "RemoveContainer" containerID="52436710a11d92761130c1ac7c74446cd372e87c51efc0ed7800b13d83b3c7d2"
Sep 10 19:53:09.398176 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:09.398137    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:53:17.401613 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:17.401572    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:53:24.799961 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:24.799920    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:53:31.900086 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:31.900041    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:53:42.203299 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:42.203265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:53:46.201835 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:46.201797    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:53:54.109191 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:53:54.109151    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:54:49.658930 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:54:49.658875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:54:49.734721 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:54:49.734680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:55:07.726227 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:07.726197    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:55:07.730630 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:07.730607    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:55:07.881660 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:07.881632    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:55:07.886504 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:07.886478    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wk98n_3eca30d4-13fd-43ee-ae7d-bb61d2f0b1b2/ovn-acl-logging/0.log"
Sep 10 19:55:08.300679 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:08.300636    2562 scope.go:117] "RemoveContainer" containerID="7e87befb8fae3838b82eb168efce9efeea0d6722dd9b0420b76ead109df236ef"
Sep 10 19:55:19.414114 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:19.414077    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:55:25.543943 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:25.543883    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:55:33.940190 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:33.940149    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:55:37.922079 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:37.922034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:55:41.311803 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:41.311767    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:55:41.999638 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:41.999601    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"]
Sep 10 19:55:42.000020 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000007    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d79696b1-fe74-4685-a4aa-40ab12471d90" containerName="payload-processing"
Sep 10 19:55:42.000078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000022    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d79696b1-fe74-4685-a4aa-40ab12471d90" containerName="payload-processing"
Sep 10 19:55:42.000078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000050    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy"
Sep 10 19:55:42.000078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000056    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a4a27c-99fb-48cb-9f93-996855f036bd" containerName="istio-proxy"
Sep 10 19:55:42.000078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000062    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b22179c-1ee1-40f3-94ed-9d14479f29ff" containerName="revoke-keys"
Sep 10 19:55:42.000078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000068    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b22179c-1ee1-40f3-94ed-9d14479f29ff" containerName="revoke-keys"
Sep 10 19:55:42.000257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000123    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b22179c-1ee1-40f3-94ed-9d14479f29ff" containerName="revoke-keys"
Sep 10 19:55:42.000257 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.000133    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d79696b1-fe74-4685-a4aa-40ab12471d90" containerName="payload-processing"
Sep 10 19:55:42.002983 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.002962    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.005922 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.005898    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-fd3007ef-kserve-self-signed-certs\""
Sep 10 19:55:42.017313 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.017282    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"]
Sep 10 19:55:42.032164 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.032116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:55:42.113336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.112970    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-model-cache\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.113336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.113023    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29v8x\" (UniqueName: \"kubernetes.io/projected/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-kube-api-access-29v8x\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.113336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.113071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tls-certs\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.113336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.113114    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tmp-dir\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.113336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.113151    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-home\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.113336 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.113191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-dshm\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.214558 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.214509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-model-cache\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.214558 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.214554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29v8x\" (UniqueName: \"kubernetes.io/projected/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-kube-api-access-29v8x\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.214788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.214681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tls-certs\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.214788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.214735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tmp-dir\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.214788 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.214773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-home\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.214959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.214810    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-dshm\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.215016 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.214992    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-model-cache\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.215132 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.215117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tmp-dir\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.215264 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.215244    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-home\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.217044 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.217021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-dshm\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.217349 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.217330    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tls-certs\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.223410 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.223386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29v8x\" (UniqueName: \"kubernetes.io/projected/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-kube-api-access-29v8x\") pod \"e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") " pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.312307 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.312213    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:42.600393 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.600364    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"]
Sep 10 19:55:42.602244 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:55:42.602209    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86d7f9ab_b18c_46e0_9d28_78a5a7daf3b5.slice/crio-92a36728f68a3580424d63df42d70c2c914cbfd84829da29653fb881c9c36589 WatchSource:0}: Error finding container 92a36728f68a3580424d63df42d70c2c914cbfd84829da29653fb881c9c36589: Status 404 returned error can't find the container with id 92a36728f68a3580424d63df42d70c2c914cbfd84829da29653fb881c9c36589
Sep 10 19:55:42.979412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.979367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp" event={"ID":"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5","Type":"ContainerStarted","Data":"3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff"}
Sep 10 19:55:42.979587 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.979418    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp" event={"ID":"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5","Type":"ContainerStarted","Data":"92a36728f68a3580424d63df42d70c2c914cbfd84829da29653fb881c9c36589"}
Sep 10 19:55:42.999269 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:42.999211    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp" podStartSLOduration=1.999194663 podStartE2EDuration="1.999194663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:55:42.998282274 +0000 UTC m=+2135.938316108" watchObservedRunningTime="2026-09-10 19:55:42.999194663 +0000 UTC m=+2135.939228495"
Sep 10 19:55:49.995539 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:49.995494    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:55:52.312723 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:52.312674    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:52.313315 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:52.312744    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:52.325852 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:52.325822    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:55:53.044228 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:55:53.044195    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:56:06.480396 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:06.480288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:56:08.185368 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:08.185334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:56:08.409300 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:08.409266    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"]
Sep 10 19:56:08.428963 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:08.428924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:56:08.808226 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:08.808185    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:56:17.503157 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:17.503118    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:56:23.429085 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.429044    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp" podUID="86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" containerName="main" containerID="cri-o://3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff" gracePeriod=45
Sep 10 19:56:23.680603 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.680537    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:56:23.810939 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.810875    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-home\") pod \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") "
Sep 10 19:56:23.811143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.810995    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-dshm\") pod \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") "
Sep 10 19:56:23.811143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811089    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-model-cache\") pod \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") "
Sep 10 19:56:23.811143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811124    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29v8x\" (UniqueName: \"kubernetes.io/projected/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-kube-api-access-29v8x\") pod \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") "
Sep 10 19:56:23.811143 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811128    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-home" (OuterVolumeSpecName: "home") pod "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" (UID: "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:56:23.811374 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811162    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tmp-dir\") pod \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") "
Sep 10 19:56:23.811374 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811278    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tls-certs\") pod \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\" (UID: \"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5\") "
Sep 10 19:56:23.811476 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811374    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" (UID: "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:56:23.811476 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811390    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" (UID: "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:56:23.811634 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811618    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-model-cache\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:23.811688 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811639    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tmp-dir\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:23.811688 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.811653    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-home\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:23.813260 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.813231    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-dshm" (OuterVolumeSpecName: "dshm") pod "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" (UID: "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:56:23.813260 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.813253    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-kube-api-access-29v8x" (OuterVolumeSpecName: "kube-api-access-29v8x") pod "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" (UID: "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5"). InnerVolumeSpecName "kube-api-access-29v8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:56:23.813436 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.813303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" (UID: "86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:56:23.912843 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.912802    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-tls-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:23.912843 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.912836    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-dshm\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:23.912843 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:23.912850    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29v8x\" (UniqueName: \"kubernetes.io/projected/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5-kube-api-access-29v8x\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:24.183045 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.183007    2562 generic.go:356] "Generic (PLEG): container finished" podID="86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" containerID="3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff" exitCode=0
Sep 10 19:56:24.183248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.183071    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"
Sep 10 19:56:24.183248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.183087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp" event={"ID":"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5","Type":"ContainerDied","Data":"3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff"}
Sep 10 19:56:24.183248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.183133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp" event={"ID":"86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5","Type":"ContainerDied","Data":"92a36728f68a3580424d63df42d70c2c914cbfd84829da29653fb881c9c36589"}
Sep 10 19:56:24.183248 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.183150    2562 scope.go:117] "RemoveContainer" containerID="3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff"
Sep 10 19:56:24.193256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.193236    2562 scope.go:117] "RemoveContainer" containerID="3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff"
Sep 10 19:56:24.193601 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:56:24.193572    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff\": container with ID starting with 3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff not found: ID does not exist" containerID="3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff"
Sep 10 19:56:24.193651 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.193612    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff"} err="failed to get container status \"3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff\": rpc error: code = NotFound desc = could not find container \"3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff\": container with ID starting with 3e3d4989f51b91e75006916b341907e11e94a1960f8864956c0f12eb7d8bcbff not found: ID does not exist"
Sep 10 19:56:24.210133 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.210101    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"]
Sep 10 19:56:24.211700 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.211679    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-fd3007ef-kserve-77dc6b776c-zksqp"]
Sep 10 19:56:24.903291 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:24.903258    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:56:25.696568 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:25.696532    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" path="/var/lib/kubelet/pods/86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5/volumes"
Sep 10 19:56:31.709381 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.709344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:56:31.955061 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.955028    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"]
Sep 10 19:56:31.955412 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.955399    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" containerName="main"
Sep 10 19:56:31.955460 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.955414    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" containerName="main"
Sep 10 19:56:31.955499 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.955489    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d7f9ab-b18c-46e0-9d28-78a5a7daf3b5" containerName="main"
Sep 10 19:56:31.960152 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.960088    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:31.963421 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.963396    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:56:31.963695 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.963679    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 19:56:31.964568 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.964550    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 19:56:31.964856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.964840    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5cfmg\""
Sep 10 19:56:31.964973 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.964876    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 19:56:31.971310 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:31.971284    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"]
Sep 10 19:56:32.087716 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.087674    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-maas-api-tls\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.087716 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.087720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-metrics-certs\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.088043 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.087754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l2gx\" (UniqueName: \"kubernetes.io/projected/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-kube-api-access-7l2gx\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.088043 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.087816    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-openshift-service-ca\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.188671 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.188628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l2gx\" (UniqueName: \"kubernetes.io/projected/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-kube-api-access-7l2gx\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.188856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.188704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-openshift-service-ca\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.188856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.188734    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-maas-api-tls\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.188856 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.188762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-metrics-certs\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.189446 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.189422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-openshift-service-ca\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.191413 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.191388    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-metrics-certs\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.191508 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.191388    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-maas-api-tls\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.197654 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.197633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l2gx\" (UniqueName: \"kubernetes.io/projected/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-kube-api-access-7l2gx\") pod \"maas-api-54b46465b9-bdkks\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.271528 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.271434    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:32.546131 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.546106    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"]
Sep 10 19:56:32.548133 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:56:32.548102    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041f7d72_ccda_4cac_83fe_8efdf8e1bc28.slice/crio-7c929ab4171dcb29de0fe7ef25bf7efb79ac6be5be9cac84943f5564807150c8 WatchSource:0}: Error finding container 7c929ab4171dcb29de0fe7ef25bf7efb79ac6be5be9cac84943f5564807150c8: Status 404 returned error can't find the container with id 7c929ab4171dcb29de0fe7ef25bf7efb79ac6be5be9cac84943f5564807150c8
Sep 10 19:56:32.549551 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:32.549527    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:56:33.231051 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:33.231008    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks" event={"ID":"041f7d72-ccda-4cac-83fe-8efdf8e1bc28","Type":"ContainerStarted","Data":"7c929ab4171dcb29de0fe7ef25bf7efb79ac6be5be9cac84943f5564807150c8"}
Sep 10 19:56:35.243065 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:35.242962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks" event={"ID":"041f7d72-ccda-4cac-83fe-8efdf8e1bc28","Type":"ContainerStarted","Data":"694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63"}
Sep 10 19:56:35.243433 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:35.243088    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:35.264565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:35.264512    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks" podStartSLOduration=1.850009545 podStartE2EDuration="4.264493186s" totalImagesPullingTime="2.414483641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:56:35.263414733 +0000 UTC m=+2188.203448566" watchObservedRunningTime="2026-09-10 19:56:35.264493186 +0000 UTC m=+2188.204527019"
Sep 10 19:56:39.838017 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:39.837979    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"]
Sep 10 19:56:39.838563 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:39.838167    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks" podUID="041f7d72-ccda-4cac-83fe-8efdf8e1bc28" containerName="maas-api" containerID="cri-o://694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63" gracePeriod=30
Sep 10 19:56:39.908676 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:39.908635    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:56:40.094595 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.094532    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:40.268193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.268157    2562 generic.go:356] "Generic (PLEG): container finished" podID="041f7d72-ccda-4cac-83fe-8efdf8e1bc28" containerID="694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63" exitCode=0
Sep 10 19:56:40.268395 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.268201    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"
Sep 10 19:56:40.268395 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.268218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks" event={"ID":"041f7d72-ccda-4cac-83fe-8efdf8e1bc28","Type":"ContainerDied","Data":"694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63"}
Sep 10 19:56:40.268395 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.268271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-bdkks" event={"ID":"041f7d72-ccda-4cac-83fe-8efdf8e1bc28","Type":"ContainerDied","Data":"7c929ab4171dcb29de0fe7ef25bf7efb79ac6be5be9cac84943f5564807150c8"}
Sep 10 19:56:40.268395 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.268299    2562 scope.go:117] "RemoveContainer" containerID="694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63"
Sep 10 19:56:40.273062 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.273041    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-openshift-service-ca\") pod \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") "
Sep 10 19:56:40.273177 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.273093    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l2gx\" (UniqueName: \"kubernetes.io/projected/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-kube-api-access-7l2gx\") pod \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") "
Sep 10 19:56:40.273245 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.273174    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-maas-api-tls\") pod \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") "
Sep 10 19:56:40.273357 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.273331    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-metrics-certs\") pod \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\" (UID: \"041f7d72-ccda-4cac-83fe-8efdf8e1bc28\") "
Sep 10 19:56:40.273432 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.273411    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "041f7d72-ccda-4cac-83fe-8efdf8e1bc28" (UID: "041f7d72-ccda-4cac-83fe-8efdf8e1bc28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:56:40.273649 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.273629    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-openshift-service-ca\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:40.275373 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.275344    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "041f7d72-ccda-4cac-83fe-8efdf8e1bc28" (UID: "041f7d72-ccda-4cac-83fe-8efdf8e1bc28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:56:40.275480 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.275403    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-kube-api-access-7l2gx" (OuterVolumeSpecName: "kube-api-access-7l2gx") pod "041f7d72-ccda-4cac-83fe-8efdf8e1bc28" (UID: "041f7d72-ccda-4cac-83fe-8efdf8e1bc28"). InnerVolumeSpecName "kube-api-access-7l2gx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:56:40.275657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.275641    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "041f7d72-ccda-4cac-83fe-8efdf8e1bc28" (UID: "041f7d72-ccda-4cac-83fe-8efdf8e1bc28"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:56:40.277816 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.277795    2562 scope.go:117] "RemoveContainer" containerID="694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63"
Sep 10 19:56:40.278228 ip-10-0-139-35 kubenswrapper[2562]: E0910 19:56:40.278206    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63\": container with ID starting with 694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63 not found: ID does not exist" containerID="694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63"
Sep 10 19:56:40.278317 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.278242    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63"} err="failed to get container status \"694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63\": rpc error: code = NotFound desc = could not find container \"694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63\": container with ID starting with 694c6c3ca7e98237f47a9283323556412990a161c3014836cc1ffc9e41fcbc63 not found: ID does not exist"
Sep 10 19:56:40.374854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.374763    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l2gx\" (UniqueName: \"kubernetes.io/projected/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-kube-api-access-7l2gx\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:40.374854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.374796    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-maas-api-tls\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:40.374854 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.374806    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/041f7d72-ccda-4cac-83fe-8efdf8e1bc28-metrics-certs\") on node \"ip-10-0-139-35.ec2.internal\" DevicePath \"\""
Sep 10 19:56:40.590367 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.590328    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"]
Sep 10 19:56:40.594797 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:40.594763    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-bdkks"]
Sep 10 19:56:41.695896 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:41.695848    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="041f7d72-ccda-4cac-83fe-8efdf8e1bc28" path="/var/lib/kubelet/pods/041f7d72-ccda-4cac-83fe-8efdf8e1bc28/volumes"
Sep 10 19:56:46.801418 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:46.801380    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:56:52.811250 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:52.811213    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:56:59.701263 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:56:59.701227    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5b7nw"]
Sep 10 19:57:02.076505 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:02.076462    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c5578f46b-w9q8q_9d8adf50-e1d5-49de-8b32-ed5f44e5f382/authorino/0.log"
Sep 10 19:57:08.330158 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:08.330121    2562 scope.go:117] "RemoveContainer" containerID="55070e3ad8ddeef31c9952342d8a451aaf772618a3c9e3cb2a61c15bae6dc958"
Sep 10 19:57:08.585338 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:08.585253    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcddd787-cxwdw_5ecc8f01-fb7f-4b67-9006-19c705c8d59f/copy-manifests/0.log"
Sep 10 19:57:08.592831 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:08.592798    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcddd787-cxwdw_5ecc8f01-fb7f-4b67-9006-19c705c8d59f/manager/0.log"
Sep 10 19:57:10.233450 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:10.233420    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c5578f46b-w9q8q_9d8adf50-e1d5-49de-8b32-ed5f44e5f382/authorino/0.log"
Sep 10 19:57:10.611703 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:10.611622    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-g9khm_6f042119-a3cc-43f2-b6cf-080645b88822/kuadrant-console-plugin/0.log"
Sep 10 19:57:10.878193 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:10.878102    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-5b7nw_395914f3-96fc-4949-9fc1-e95f364e37a7/limitador/0.log"
Sep 10 19:57:11.764202 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:11.764171    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cf7f4d9c5-k6lcz_e26c5c4f-a1e7-4762-ba4d-9066eae20633/kube-auth-proxy/0.log"
Sep 10 19:57:12.282857 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:12.282821    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9dc57874b-zzp9h_fc444d45-54f7-4c61-a08d-34658072f79a/router/0.log"
Sep 10 19:57:13.001538 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:13.001511    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5kscj_7c06fd0f-d1d9-4a0c-bd2e-204cfcacfd0a/main/0.log"
Sep 10 19:57:13.240882 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:13.240844    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dvjcp_c22612dd-8653-4332-b7ae-b595287342a0/main/0.log"
Sep 10 19:57:18.736402 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.736365    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lkczj/must-gather-b6k5k"]
Sep 10 19:57:18.736823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.736738    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="041f7d72-ccda-4cac-83fe-8efdf8e1bc28" containerName="maas-api"
Sep 10 19:57:18.736823 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.736749    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="041f7d72-ccda-4cac-83fe-8efdf8e1bc28" containerName="maas-api"
Sep 10 19:57:18.736927 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.736830    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="041f7d72-ccda-4cac-83fe-8efdf8e1bc28" containerName="maas-api"
Sep 10 19:57:18.739719 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.739700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:18.742553 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.742523    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lkczj\"/\"kube-root-ca.crt\""
Sep 10 19:57:18.742689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.742576    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lkczj\"/\"openshift-service-ca.crt\""
Sep 10 19:57:18.742689 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.742667    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lkczj\"/\"default-dockercfg-rr7jr\""
Sep 10 19:57:18.751173 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.751144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:57:18.752565 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.752544    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lkczj/must-gather-b6k5k"]
Sep 10 19:57:18.822298 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.822258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log" limit=1024
Sep 10 19:57:18.930078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.930038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-582nz\" (UniqueName: \"kubernetes.io/projected/4e8f587a-d991-49d7-82ef-4f8d66c1b7fe-kube-api-access-582nz\") pod \"must-gather-b6k5k\" (UID: \"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe\") " pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:18.930078 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:18.930076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4e8f587a-d991-49d7-82ef-4f8d66c1b7fe-must-gather-output\") pod \"must-gather-b6k5k\" (UID: \"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe\") " pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:19.031244 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:19.031141    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-582nz\" (UniqueName: \"kubernetes.io/projected/4e8f587a-d991-49d7-82ef-4f8d66c1b7fe-kube-api-access-582nz\") pod \"must-gather-b6k5k\" (UID: \"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe\") " pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:19.031244 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:19.031180    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4e8f587a-d991-49d7-82ef-4f8d66c1b7fe-must-gather-output\") pod \"must-gather-b6k5k\" (UID: \"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe\") " pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:19.031596 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:19.031573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4e8f587a-d991-49d7-82ef-4f8d66c1b7fe-must-gather-output\") pod \"must-gather-b6k5k\" (UID: \"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe\") " pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:19.040763 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:19.040729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-582nz\" (UniqueName: \"kubernetes.io/projected/4e8f587a-d991-49d7-82ef-4f8d66c1b7fe-kube-api-access-582nz\") pod \"must-gather-b6k5k\" (UID: \"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe\") " pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:19.049194 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:19.049155    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lkczj/must-gather-b6k5k"
Sep 10 19:57:19.337959 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:19.337933    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lkczj/must-gather-b6k5k"]
Sep 10 19:57:19.340274 ip-10-0-139-35 kubenswrapper[2562]: W0910 19:57:19.340245    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e8f587a_d991_49d7_82ef_4f8d66c1b7fe.slice/crio-34cb12126f50640c12e06010114f2658cfad5a652585d0c0f29ced65a55f9d69 WatchSource:0}: Error finding container 34cb12126f50640c12e06010114f2658cfad5a652585d0c0f29ced65a55f9d69: Status 404 returned error can't find the container with id 34cb12126f50640c12e06010114f2658cfad5a652585d0c0f29ced65a55f9d69
Sep 10 19:57:19.465564 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:19.465527    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lkczj/must-gather-b6k5k" event={"ID":"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe","Type":"ContainerStarted","Data":"34cb12126f50640c12e06010114f2658cfad5a652585d0c0f29ced65a55f9d69"}
Sep 10 19:57:21.479629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:21.479592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lkczj/must-gather-b6k5k" event={"ID":"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe","Type":"ContainerStarted","Data":"ab563698d85c304786d9c0e2f255620d23d378ba055adc1e70cd2c3867be7deb"}
Sep 10 19:57:21.479629 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:21.479639    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lkczj/must-gather-b6k5k" event={"ID":"4e8f587a-d991-49d7-82ef-4f8d66c1b7fe","Type":"ContainerStarted","Data":"7862b97a3c0f6d6529fca397965c3e32e45b690fa589aeeb41167e5316714e40"}
Sep 10 19:57:21.497296 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:21.497233    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lkczj/must-gather-b6k5k" podStartSLOduration=2.354381129 podStartE2EDuration="3.497211714s" totalImagesPullingTime="1.142830585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:57:21.496690369 +0000 UTC m=+2234.436724194" watchObservedRunningTime="2026-09-10 19:57:21.497211714 +0000 UTC m=+2234.437245548"
Sep 10 19:57:22.214291 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:22.214250    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nhl6m_e50d4c91-53ee-4496-9854-3a3abb0a1ca7/global-pull-secret-syncer/0.log"
Sep 10 19:57:22.270719 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:22.270675    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7rxpl_3c71f7e5-908d-4f2d-b087-2a5adb55ae5b/konnectivity-agent/0.log"
Sep 10 19:57:22.411626 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:22.411597    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-35.ec2.internal_50f4e7fca0ff25f9b39860d2874f4285/haproxy/0.log"
Sep 10 19:57:27.365817 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:27.365776    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c5578f46b-w9q8q_9d8adf50-e1d5-49de-8b32-ed5f44e5f382/authorino/0.log"
Sep 10 19:57:27.522698 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:27.522665    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-g9khm_6f042119-a3cc-43f2-b6cf-080645b88822/kuadrant-console-plugin/0.log"
Sep 10 19:57:27.626932 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:27.626824    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-5b7nw_395914f3-96fc-4949-9fc1-e95f364e37a7/limitador/0.log"
Sep 10 19:57:29.518556 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:29.518517    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-sk5q6_e7746c37-e94a-4efd-a1f9-e4ad8e13ff9d/cluster-monitoring-operator/0.log"
Sep 10 19:57:29.806159 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:29.806108    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-lkvxj_2437fe2e-7e58-4ee8-8247-e53eb4f69746/monitoring-plugin/0.log"
Sep 10 19:57:30.290638 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.290530    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wt4nz_d96f0961-f8a6-460e-9f0a-22eb7c39b135/node-exporter/0.log"
Sep 10 19:57:30.344510 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.344476    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wt4nz_d96f0961-f8a6-460e-9f0a-22eb7c39b135/kube-rbac-proxy/0.log"
Sep 10 19:57:30.388259 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.388205    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wt4nz_d96f0961-f8a6-460e-9f0a-22eb7c39b135/init-textfile/0.log"
Sep 10 19:57:30.444091 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.444054    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4grqx_f98689ff-21ff-43fc-973a-325ae09b957c/kube-rbac-proxy-main/0.log"
Sep 10 19:57:30.493497 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.493462    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4grqx_f98689ff-21ff-43fc-973a-325ae09b957c/kube-rbac-proxy-self/0.log"
Sep 10 19:57:30.537174 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.537139    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4grqx_f98689ff-21ff-43fc-973a-325ae09b957c/openshift-state-metrics/0.log"
Sep 10 19:57:30.610547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.610463    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a313236a-1122-4ffb-8d09-781be18c2a15/prometheus/0.log"
Sep 10 19:57:30.671211 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.671179    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a313236a-1122-4ffb-8d09-781be18c2a15/config-reloader/0.log"
Sep 10 19:57:30.717606 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.717559    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a313236a-1122-4ffb-8d09-781be18c2a15/thanos-sidecar/0.log"
Sep 10 19:57:30.750031 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.750003    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a313236a-1122-4ffb-8d09-781be18c2a15/kube-rbac-proxy-web/0.log"
Sep 10 19:57:30.801244 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.801216    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a313236a-1122-4ffb-8d09-781be18c2a15/kube-rbac-proxy/0.log"
Sep 10 19:57:30.843731 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.843699    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a313236a-1122-4ffb-8d09-781be18c2a15/kube-rbac-proxy-thanos/0.log"
Sep 10 19:57:30.884877 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.884788    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a313236a-1122-4ffb-8d09-781be18c2a15/init-config-reloader/0.log"
Sep 10 19:57:30.930408 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.930372    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ddmn2_85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4/prometheus-operator/0.log"
Sep 10 19:57:30.956172 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:30.956140    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ddmn2_85ff17f3-fb2b-4cb1-a2ce-1ea3cd4fe9c4/kube-rbac-proxy/0.log"
Sep 10 19:57:32.424687 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.424650    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"]
Sep 10 19:57:32.431833 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.431802    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.450531 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.450484    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"]
Sep 10 19:57:32.506611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.506576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnpfv\" (UniqueName: \"kubernetes.io/projected/18140705-f657-4428-ae96-56c8dde63ba2-kube-api-access-vnpfv\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.506611 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.506620    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-lib-modules\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.506853 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.506642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-proc\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.506853 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.506827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-podres\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.506951 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.506879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-sys\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.573095 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.573064    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-wqgz2_04f372b4-a549-4f39-b87f-7f53a360a242/networking-console-plugin/0.log"
Sep 10 19:57:32.608233 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-podres\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.608443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-sys\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.608443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608385    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-sys\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.608443 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnpfv\" (UniqueName: \"kubernetes.io/projected/18140705-f657-4428-ae96-56c8dde63ba2-kube-api-access-vnpfv\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.608646 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-lib-modules\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.608703 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608664    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-proc\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.608863 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-podres\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.609191 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.608939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-proc\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.609305 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.609290    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18140705-f657-4428-ae96-56c8dde63ba2-lib-modules\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.620393 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.620355    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnpfv\" (UniqueName: \"kubernetes.io/projected/18140705-f657-4428-ae96-56c8dde63ba2-kube-api-access-vnpfv\") pod \"perf-node-gather-daemonset-pl72n\" (UID: \"18140705-f657-4428-ae96-56c8dde63ba2\") " pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:32.748171 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:32.748077    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:33.043807 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:33.043714    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"]
Sep 10 19:57:33.425737 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:33.425697    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/1.log"
Sep 10 19:57:33.433075 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:33.433036    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7b5pk_2f2e6b2f-afbe-45bb-84bc-582ace2e0379/console-operator/2.log"
Sep 10 19:57:33.572345 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:33.572297    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n" event={"ID":"18140705-f657-4428-ae96-56c8dde63ba2","Type":"ContainerStarted","Data":"8676bf3698f9e951d2e0f188b69eb174d93345ef65beaa57e054ab88a5831a90"}
Sep 10 19:57:33.572547 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:33.572378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n" event={"ID":"18140705-f657-4428-ae96-56c8dde63ba2","Type":"ContainerStarted","Data":"a36919b9d57c75ce841effc1299624c1e675d696e54bf0f0d7a1fe8011da5c8a"}
Sep 10 19:57:33.574242 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:33.573285    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:33.595210 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:33.595138    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n" podStartSLOduration=1.595114097 podStartE2EDuration="1.595114097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:57:33.593519441 +0000 UTC m=+2246.533553299" watchObservedRunningTime="2026-09-10 19:57:33.595114097 +0000 UTC m=+2246.535147931"
Sep 10 19:57:34.835270 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:34.835235    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-cv7hn_ccd4a7aa-770f-4dba-a34d-490ba52734b6/volume-data-source-validator/0.log"
Sep 10 19:57:36.056586 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:36.056525    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nh2g2_c6f9c1d8-a78a-4541-bb95-e368edfee6fe/dns/0.log"
Sep 10 19:57:36.088822 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:36.088787    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nh2g2_c6f9c1d8-a78a-4541-bb95-e368edfee6fe/kube-rbac-proxy/0.log"
Sep 10 19:57:36.170546 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:36.170518    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2htct_491040f7-5239-4e74-ac39-1b5a8190e333/dns-node-resolver/0.log"
Sep 10 19:57:36.897664 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:36.897632    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bhgv7_05dde958-85c8-4751-87d7-cd7b3d9314cc/node-ca/0.log"
Sep 10 19:57:38.334686 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:38.334647    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cf7f4d9c5-k6lcz_e26c5c4f-a1e7-4762-ba4d-9066eae20633/kube-auth-proxy/0.log"
Sep 10 19:57:38.478185 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:38.478144    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9dc57874b-zzp9h_fc444d45-54f7-4c61-a08d-34658072f79a/router/0.log"
Sep 10 19:57:39.402801 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:39.402759    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rkxv6_8fa87b08-6c57-4461-a536-f2a9adeac807/serve-healthcheck-canary/0.log"
Sep 10 19:57:39.957903 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:39.957856    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-s4d5w_20d869f4-b859-45f8-a21d-d90bf157087f/insights-operator/1.log"
Sep 10 19:57:39.958283 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:39.958256    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-s4d5w_20d869f4-b859-45f8-a21d-d90bf157087f/insights-operator/0.log"
Sep 10 19:57:39.983181 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:39.983149    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cn5vq_1bb7defd-3c95-44ee-956c-6e6e567622aa/kube-rbac-proxy/0.log"
Sep 10 19:57:40.013308 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:40.013267    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cn5vq_1bb7defd-3c95-44ee-956c-6e6e567622aa/exporter/0.log"
Sep 10 19:57:40.041256 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:40.041218    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cn5vq_1bb7defd-3c95-44ee-956c-6e6e567622aa/extractor/0.log"
Sep 10 19:57:40.597970 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:40.597942    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lkczj/perf-node-gather-daemonset-pl72n"
Sep 10 19:57:43.189168 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:43.189122    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcddd787-cxwdw_5ecc8f01-fb7f-4b67-9006-19c705c8d59f/manager/0.log"
Sep 10 19:57:43.215266 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:43.215238    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ddcddd787-cxwdw_5ecc8f01-fb7f-4b67-9006-19c705c8d59f/copy-manifests/0.log"
Sep 10 19:57:44.194451 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:44.194397    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cf7f4d9c5-k6lcz_e26c5c4f-a1e7-4762-ba4d-9066eae20633/kube-auth-proxy/0.log"
Sep 10 19:57:44.295039 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:44.295008    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9dc57874b-zzp9h_fc444d45-54f7-4c61-a08d-34658072f79a/router/0.log"
Sep 10 19:57:44.628513 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:44.628405    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57fccbbc8-gh4wn_30162097-2698-4b58-a980-b4f26d4e5793/manager/0.log"
Sep 10 19:57:46.116299 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:46.116230    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cf7f4d9c5-k6lcz_e26c5c4f-a1e7-4762-ba4d-9066eae20633/kube-auth-proxy/0.log"
Sep 10 19:57:46.222812 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:46.222775    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9dc57874b-zzp9h_fc444d45-54f7-4c61-a08d-34658072f79a/router/0.log"
Sep 10 19:57:50.541472 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:50.541422    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-8szwh_1a7b5479-66fb-4a0a-83d6-1a50656b07af/kube-storage-version-migrator-operator/1.log"
Sep 10 19:57:50.543657 ip-10-0-139-35 kubenswrapper[2562]: I0910 19:57:50.543626    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-8szwh_1a7b5479-66fb-4a0a-83d6-1a50656b07af/kube-storage-version-migrator-operator/0.log"