Sep 11 23:01:24.662210 ip-10-0-138-44 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 23:01:25.102123 ip-10-0-138-44 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 23:01:25.102123 ip-10-0-138-44 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 23:01:25.102123 ip-10-0-138-44 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 23:01:25.102123 ip-10-0-138-44 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 23:01:25.102123 ip-10-0-138-44 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 23:01:25.104375 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.104266    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 23:01:25.108618 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108602    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:25.108618 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108617    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108621    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108625    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108628    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108634    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108639    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108642    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108645    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108648    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108651    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108654    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108657    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108660    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108663    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108665    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108668    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108670    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108673    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108677    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:25.108680 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108681    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108685    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108688    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108691    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108694    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108697    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108700    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108703    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108706    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108709    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108711    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108714    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108716    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108719    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108722    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108724    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108727    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108730    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108733    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108735    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:25.109155 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108740    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108744    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108748    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108751    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108753    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108756    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108759    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108761    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108764    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108766    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108769    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108772    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108776    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108780    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108783    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108785    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108789    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108792    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108794    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108797    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:25.109639 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108799    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108802    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108805    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108807    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108810    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108814    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108817    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108819    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108822    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108825    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108828    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108831    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108833    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108836    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108839    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108842    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108844    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108847    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108849    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108852    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:25.110181 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108862    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108866    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108869    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108872    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108874    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108877    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108880    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108882    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108885    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108887    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108890    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108893    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108896    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108898    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108901    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108903    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108906    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108908    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.108924    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109662    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:25.110772 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109668    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109671    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109674    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109678    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109681    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109683    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109686    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109689    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109693    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109695    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109698    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109700    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109703    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109706    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109709    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109712    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109716    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109720    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109723    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:25.111441 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109726    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109729    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109732    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109735    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109738    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109740    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109743    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109746    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109748    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109751    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109754    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109756    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109759    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109761    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109764    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109766    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109769    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109772    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109774    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109778    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:25.111927 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109781    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109783    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109786    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109789    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109791    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109794    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109797    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109799    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109802    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109806    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109808    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109811    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109813    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109816    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109819    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109821    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109824    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109826    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109829    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:25.112421 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109831    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109834    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109837    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109839    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109842    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109844    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109847    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109850    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109852    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109855    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109857    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109860    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109863    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109866    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109868    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109871    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109874    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109876    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109881    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109883    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:25.112896 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109886    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109889    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109892    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109894    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109897    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109899    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109902    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109904    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109907    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109926    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109931    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109934    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109937    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109940    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109943    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109945    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109948    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109952    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109955    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:25.113400 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.109959    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.110346    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.110351    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.110355    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.110400    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111617    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111738    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111744    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111747    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111751    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111755    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111758    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111761    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111764    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111768    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111771    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111774    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111777    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111779    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111782    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:25.113859 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111785    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111787    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111790    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111793    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111795    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111798    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111801    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111803    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111806    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111808    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111811    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111813    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111816    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111818    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111821    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111824    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111826    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111829    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111835    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:25.114360 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111840    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111844    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111847    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111852    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111855    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111858    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111861    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111864    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111867    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111870    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111873    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111875    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111878    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111881    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111883    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111886    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111888    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111891    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111893    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111896    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:25.114821 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111898    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111901    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111903    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111905    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111908    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111927    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111930    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111933    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111936    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111939    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111942    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111945    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111948    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111951    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111954    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111958    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111960    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111963    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111966    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111969    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:25.115359 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111971    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111976    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111979    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111983    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111985    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111988    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111991    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111994    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.111998    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112001    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112004    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112007    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112010    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112012    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112015    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112017    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112020    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112023    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112026    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:25.115850 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112028    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112031    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112107    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112146    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112153    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112157    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112162    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112165    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112169    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112173    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112177    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112180    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112183    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112187    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112190    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112193    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112196    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112199    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112202    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112205    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112211    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112214    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112217    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112221    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 23:01:25.116417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112224    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112228    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112231    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112234    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112238    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112241    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112244    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112247    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112250    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112253    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112258    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112261    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112264    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112267    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112270    2580 flags.go:64] FLAG: --enable-server="true"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112273    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112277    2580 flags.go:64] FLAG: --event-burst="100"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112280    2580 flags.go:64] FLAG: --event-qps="50"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112283    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112286    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112289    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112293    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112295    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112298    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112301    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 11 23:01:25.117031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112304    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112307    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112310    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112313    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112316    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112319    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112322    2580 flags.go:64] FLAG: --feature-gates=""
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112326    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112329    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112332    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112335    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112338    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112341    2580 flags.go:64] FLAG: --help="false"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112344    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112347    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112350    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112353    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112357    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112361    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112364    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112367    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112370    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112373    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 23:01:25.117613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112376    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112379    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112382    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112385    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112388    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112391    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112393    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112396    2580 flags.go:64] FLAG: --lock-file=""
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112399    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112402    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112405    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112410    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112413    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112416    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112419    2580 flags.go:64] FLAG: --logging-format="text"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112422    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112426    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112430    2580 flags.go:64] FLAG: --manifest-url=""
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112433    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112438    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112442    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112446    2580 flags.go:64] FLAG: --max-pods="110"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112449    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112452    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112455    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 23:01:25.118222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112458    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112461    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112464    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112467    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112475    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112477    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112480    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112484    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112487    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112492    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112495    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112498    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112501    2580 flags.go:64] FLAG: --port="10250"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112504    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112507    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-018b1106370b0f113"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112510    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112513    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112516    2580 flags.go:64] FLAG: --register-node="true"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112519    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112522    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112525    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112528    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112530    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112534    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112537    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 23:01:25.118869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112540    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112543    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112546    2580 flags.go:64] FLAG: --runonce="false"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112549    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112552    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112555    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112558    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112561    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112564    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112567    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112570    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112572    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112575    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112578    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112580    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112583    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112586    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112589    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112595    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112597    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112600    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112605    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112608    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112611    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112613    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 23:01:25.119499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112616    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112619    2580 flags.go:64] FLAG: --v="2"
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112626    2580 flags.go:64] FLAG: --version="false"
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112630    2580 flags.go:64] FLAG: --vmodule=""
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112634    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.112637    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112752    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112756    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112759    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112762    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112765    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112768    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112771    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112773    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112776    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112778    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112781    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112784    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112786    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112789    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112792    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:25.120130 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112794    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112797    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112799    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112802    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112805    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112808    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112810    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112813    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112816    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112820    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112823    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112825    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112828    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112830    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112833    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112836    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112838    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112843    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112845    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112848    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:25.120653 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112851    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112854    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112866    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112869    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112871    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112874    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112877    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112879    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112882    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112885    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112887    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112890    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112892    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112895    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112898    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112900    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112903    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112906    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112927    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112930    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:25.121281 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112934    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112938    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112941    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112944    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112947    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112950    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112952    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112955    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112957    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112961    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112964    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112967    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112971    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112973    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112978    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112981    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112985    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112988    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112991    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:25.121782 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112995    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.112998    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113000    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113004    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113007    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113009    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113012    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113015    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113017    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113020    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113022    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113027    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113029    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113032    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113035    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113038    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113040    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113043    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113045    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113048    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:25.122291 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113051    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113053    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113057    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113060    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.113063    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.113828    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.121603    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.121618    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.122564    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.122573    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122627    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122632    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122636    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122639    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122642    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:25.122777 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122646    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122649    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122652    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122655    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122658    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122661    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122664    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122667    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122669    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122672    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122675    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122678    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122681    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122684    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122686    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122689    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122692    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122695    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122697    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122700    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:25.123170 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122703    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122706    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122709    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122712    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122716    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122719    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122722    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122725    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122727    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122730    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122733    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122735    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122738    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122741    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122743    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122748    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122752    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122755    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122758    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:25.123647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122761    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122765    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122768    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122771    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122774    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122777    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122780    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122783    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122786    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122789    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122792    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122795    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122798    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122801    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122803    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122807    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122809    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122813    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122816    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122819    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:25.124139 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122822    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122825    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122828    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122832    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122835    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122838    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122840    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122843    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122846    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122848    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122851    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122853    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122856    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122859    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122861    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122864    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122866    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122869    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122872    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:25.124627 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122875    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122877    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122880    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122883    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122886    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122888    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122891    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122894    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122897    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122899    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122903    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122909    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122925    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122929    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122932    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:25.125148 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.122935    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.122940    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123046    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123051    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123054    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123058    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123061    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123064    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123067    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123070    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123072    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123075    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123078    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123080    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123083    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:25.125512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123086    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123088    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123091    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123094    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123096    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123100    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123102    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123105    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123108    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123110    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123113    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123116    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123118    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123121    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123124    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123126    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123129    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123132    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123134    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123136    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:25.125888 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123139    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123142    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123144    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123147    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123149    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123152    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123155    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123157    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123160    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123162    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123164    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123167    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123170    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123172    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123175    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123178    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123180    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123183    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123186    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:25.126398 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123188    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123191    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123193    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123198    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123201    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123204    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123207    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123210    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123213    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123216    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123218    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123221    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123223    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123226    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123228    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123231    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123234    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123236    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123239    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:25.126848 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123242    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123245    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123247    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123250    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123252    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123255    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123258    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123260    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123263    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123265    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123268    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123271    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123274    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123277    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123280    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123283    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123286    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123288    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123291    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123294    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:25.127319 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123296    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123299    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123302    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123304    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123307    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123310    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123314    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:25.123317    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.123322    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.123422    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.125872    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.127051    2580 server.go:1019] "Starting client certificate rotation"
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.127153    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 23:01:25.127833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.127182    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 23:01:25.129813 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.129800    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 23:01:25.131939 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.131923    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 23:01:25.143555 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.143447    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 11 23:01:25.146832 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.146814    2580 log.go:25] "Validated CRI v1 image API"
Sep 11 23:01:25.148495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.148470    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 23:01:25.152744 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.152722    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b4f58f1c-65b7-4207-8672-87286540fc53:/dev/nvme0n1p3 d1d9b9fb-f9fb-417a-b02f-879e5c52184d:/dev/nvme0n1p4]
Sep 11 23:01:25.152821 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.152743    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 23:01:25.156104 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.156083    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 23:01:25.159081 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.158968    2580 manager.go:216] Machine: {Timestamp:2026-09-11 23:01:25.157512518 +0000 UTC m=+0.369483683 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091173 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22c9eace4f15893422280554e44fa7 SystemUUID:ec22c9ea-ce4f-1589-3422-280554e44fa7 BootID:55503ca2-6c2f-4143-a762-15afe2363d11 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c9:13:44:64:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c9:13:44:64:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:98:45:a3:c4:55 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 23:01:25.159081 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.159074    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 23:01:25.159213 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.159181    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 23:01:25.162792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.162770    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 23:01:25.162939 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.162796    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-44.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 23:01:25.163750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.163740    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 23:01:25.163786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.163754    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 23:01:25.163786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.163763    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 23:01:25.163786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.163778    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 23:01:25.165183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.165173    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 23:01:25.169889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.169877    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 23:01:25.170017 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170007    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 23:01:25.170049 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170038    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 23:01:25.170083 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170051    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 23:01:25.170083 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170073    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 23:01:25.170140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170086    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 11 23:01:25.170140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170099    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 23:01:25.170519 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170506    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 23:01:25.170586 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.170531    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 23:01:25.172839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.172824    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 23:01:25.175873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.175860    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 23:01:25.175935 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.175891    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 23:01:25.179759 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.179731    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 23:01:25.179836 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.179768    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d7fnk"
Sep 11 23:01:25.179836 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.179802    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 23:01:25.187406 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.187392    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d7fnk"
Sep 11 23:01:25.319605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319571    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 23:01:25.319605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319598    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 23:01:25.319605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319607    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 23:01:25.319605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319614    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319628    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319635    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319653    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319662    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319682    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319691    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319710    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 23:01:25.319847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.319718    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 23:01:25.320708 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.320696    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 23:01:25.320759 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.320712    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 23:01:25.324978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.324959    2580 server.go:1268] "Started kubelet"
Sep 11 23:01:25.325066 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.325040    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 23:01:25.325143 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.325122    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 23:01:25.325182 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.325162    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 23:01:25.326628 ip-10-0-138-44 systemd[1]: Started Kubernetes Kubelet.
Sep 11 23:01:25.327085 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.327047    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 23:01:25.332355 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.332331    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 23:01:25.333100 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.333082    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 23:01:25.336350 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.336329    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 23:01:25.337002 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.336983    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 23:01:25.337621 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.337413    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:25.337772 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.337760    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 23:01:25.337889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.337872    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 23:01:25.338007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.337846    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 23:01:25.338007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.337845    2580 factory.go:55] Registering systemd factory
Sep 11 23:01:25.338007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.337964    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 11 23:01:25.338007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.337979    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 23:01:25.338007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.337987    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 23:01:25.338287 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.338204    2580 factory.go:153] Registering CRI-O factory
Sep 11 23:01:25.338287 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.338221    2580 factory.go:223] Registration of the crio container factory successfully
Sep 11 23:01:25.338342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.338286    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 23:01:25.338342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.338304    2580 factory.go:103] Registering Raw factory
Sep 11 23:01:25.338342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.338328    2580 manager.go:1196] Started watching for new ooms in manager
Sep 11 23:01:25.338721 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.338703    2580 manager.go:349] Starting recovery of all containers
Sep 11 23:01:25.340976 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.340953    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:25.342221 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.342200    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-44.ec2.internal" not found
Sep 11 23:01:25.343315 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.343293    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-44.ec2.internal\" not found" node="ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.349150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.349133    2580 manager.go:354] Recovery completed
Sep 11 23:01:25.353677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.353614    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:25.355344 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.355329    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:25.355423 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.355356    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:25.355423 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.355367    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:25.355903 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.355890    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 23:01:25.355903 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.355899    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 23:01:25.355903 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.355903    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:25.356073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.356000    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 23:01:25.356686 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.356671    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:25.356756 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.356703    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:25.356756 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.356728    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:25.357149 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.357136    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-44.ec2.internal" not found
Sep 11 23:01:25.358223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.358211    2580 policy_none.go:49] "None policy: Start"
Sep 11 23:01:25.358256 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.358227    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 23:01:25.358256 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.358238    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 23:01:25.360171 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.360161    2580 policy_none.go:47] "Start"
Sep 11 23:01:25.390520 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.390504    2580 manager.go:337] "Starting Device Plugin manager"
Sep 11 23:01:25.390611 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.390544    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 23:01:25.390611 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.390556    2580 server.go:87] "Starting device plugin registration server"
Sep 11 23:01:25.390799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.390789    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 23:01:25.391590 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.391052    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 23:01:25.391590 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.391354    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 23:01:25.391590 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.391430    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 23:01:25.391590 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.391439    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 23:01:25.393719 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.392201    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 23:01:25.393719 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.392258    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:25.414902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.406115    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 23:01:25.414902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.407256    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 23:01:25.414902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.407309    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 23:01:25.414902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.407335    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 23:01:25.414902 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.407379    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 23:01:25.414902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.409205    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:25.414902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.411685    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-138-44.ec2.internal" not found
Sep 11 23:01:25.492095 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.492073    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:25.493034 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.493011    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:25.493131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.493043    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:25.493131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.493053    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:25.493131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.493074    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.502022 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.502009    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.502061 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.502028    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-44.ec2.internal\": node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:25.508011 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.507995    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"]
Sep 11 23:01:25.508068 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.508050    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:25.509504 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.509491    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:25.509549 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.509515    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:25.509549 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.509526    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:25.511812 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.511800    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:25.511928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.511903    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.511966 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.511942    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:25.515823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.515808    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:25.515897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.515829    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:25.515897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.515838    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:25.515897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.515865    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:25.515897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.515882    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:25.515897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.515897    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:25.517711 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.517699    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:25.518177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.518164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.518223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.518188    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:25.518895 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.518879    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:25.518979 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.518905    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:25.518979 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.518932    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:25.540462 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.540443    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-44.ec2.internal\" not found" node="ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.544849 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.544833    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-44.ec2.internal\" not found" node="ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.618131 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.618080    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:25.639353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.639329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3787794cb4c7a97e0f0378114cfc76b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal\" (UID: \"f3787794cb4c7a97e0f0378114cfc76b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.639448 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.639356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3787794cb4c7a97e0f0378114cfc76b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal\" (UID: \"f3787794cb4c7a97e0f0378114cfc76b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.639448 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.639375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/067c5e3fd43aceeec531c7141228c7cb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-44.ec2.internal\" (UID: \"067c5e3fd43aceeec531c7141228c7cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.718594 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.718577    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:25.740239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.740220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3787794cb4c7a97e0f0378114cfc76b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal\" (UID: \"f3787794cb4c7a97e0f0378114cfc76b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.740314 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.740244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3787794cb4c7a97e0f0378114cfc76b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal\" (UID: \"f3787794cb4c7a97e0f0378114cfc76b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.740314 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.740264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/067c5e3fd43aceeec531c7141228c7cb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-44.ec2.internal\" (UID: \"067c5e3fd43aceeec531c7141228c7cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.740314 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.740298    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/067c5e3fd43aceeec531c7141228c7cb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-44.ec2.internal\" (UID: \"067c5e3fd43aceeec531c7141228c7cb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.740415 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.740311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3787794cb4c7a97e0f0378114cfc76b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal\" (UID: \"f3787794cb4c7a97e0f0378114cfc76b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.740415 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.740330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3787794cb4c7a97e0f0378114cfc76b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal\" (UID: \"f3787794cb4c7a97e0f0378114cfc76b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.824032 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.823997    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:25.843780 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.843753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.849670 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:25.849539    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:25.924421 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:25.924312    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:26.025977 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:26.025948    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:26.129928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.129894    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 23:01:26.130407 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:26.129932    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:26.130407 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.130075    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 23:01:26.130407 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.130090    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 23:01:26.230198 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:26.230113    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:26.323571 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.323502    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 22:56:25 +0000 UTC" deadline="2028-06-16 08:19:40.314878036 +0000 UTC"
Sep 11 23:01:26.323571 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.323565    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15441h18m13.991315934s"
Sep 11 23:01:26.330757 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:26.330740    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-44.ec2.internal\" not found"
Sep 11 23:01:26.357500 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.357481    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:26.397674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.397642    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:26.437554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.437522    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:26.450234 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.450215    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 23:01:26.451350 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.451336    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"
Sep 11 23:01:26.459326 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.459308    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 23:01:26.474967 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:26.474937    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3787794cb4c7a97e0f0378114cfc76b.slice/crio-ba34e6d871f890a512354cf712d1751cdeb8ac3ef8f8101113921e7f4ad90f5f WatchSource:0}: Error finding container ba34e6d871f890a512354cf712d1751cdeb8ac3ef8f8101113921e7f4ad90f5f: Status 404 returned error can't find the container with id ba34e6d871f890a512354cf712d1751cdeb8ac3ef8f8101113921e7f4ad90f5f
Sep 11 23:01:26.481240 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:26.481222    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:01:27.171773 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.171738    2580 apiserver.go:52] "Watching apiserver"
Sep 11 23:01:27.180788 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.180761    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 23:01:27.184108 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.184077    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx","openshift-cluster-node-tuning-operator/tuned-dsf2b","openshift-dns/node-resolver-f9ttb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal","openshift-multus/multus-additional-cni-plugins-6hwws","openshift-multus/multus-n8m6b","openshift-multus/network-metrics-daemon-9msgs","kube-system/global-pull-secret-syncer-j2kzp","kube-system/konnectivity-agent-bnzhf","openshift-image-registry/node-ca-h2f6t","openshift-network-diagnostics/network-check-target-z7tt8","openshift-network-operator/iptables-alerter-qwbzn","openshift-ovn-kubernetes/ovnkube-node-7mn4n","kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal"]
Sep 11 23:01:27.186685 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.186650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.189062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.189003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.189792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.189598    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 23:01:27.189792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.189598    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 23:01:27.189792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.189669    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 23:01:27.189792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.189604    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-85jbp\""
Sep 11 23:01:27.193130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.192021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.193130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.192411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9f5hm\""
Sep 11 23:01:27.193130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.192472    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:01:27.193377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.193197    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 23:01:27.195605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.195582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.196102 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.195984    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 23:01:27.196363 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.196246    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 23:01:27.196432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.196419    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 23:01:27.196642 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.196620    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-94smz\""
Sep 11 23:01:27.197857 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.197840    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.199523 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.199502    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 23:01:27.199638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.199589    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 23:01:27.199638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.199598    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 23:01:27.199761 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.199602    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nlbgt\""
Sep 11 23:01:27.199761 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.199719    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 23:01:27.199865 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.199852    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 23:01:27.200257 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.200240    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:27.200353 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.200316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:27.200729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.200712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dlgt8\""
Sep 11 23:01:27.200729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.200721    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 23:01:27.202763 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.202740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.202862 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.202827    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:27.205254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.205235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.207815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.207796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.208251 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.208230    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nf7mg\""
Sep 11 23:01:27.208360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.208316    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 23:01:27.208360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.208351    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 23:01:27.210101 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.210080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:27.210199 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.210161    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:27.211298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.211146    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8g7gw\""
Sep 11 23:01:27.211298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.211146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 23:01:27.211298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.211275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 23:01:27.212688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.212667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.215310 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.215288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.215415 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.215358    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 23:01:27.215585 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.215565    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:01:27.215859 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.215839    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 23:01:27.216295 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.216277    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s2r6w\""
Sep 11 23:01:27.218183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.218165    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 23:01:27.218312 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.218262    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 23:01:27.218425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.218262    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 23:01:27.218676 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.218643    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 23:01:27.218813 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.218749    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 23:01:27.218874 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.218864    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jw4fx\""
Sep 11 23:01:27.219959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.219939    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 23:01:27.239633 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.239273    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 23:01:27.248610 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-conf-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.248723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.248723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovn-node-metrics-cert\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.248723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysctl-d\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.248723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eab07679-827b-4064-b7d4-65c86f346a51-etc-tuned\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.248901 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-kubelet\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.248901 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-cni-bin\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.248901 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/440d1a51-bd49-49b0-8566-314cf9a9bd17-agent-certs\") pod \"konnectivity-agent-bnzhf\" (UID: \"440d1a51-bd49-49b0-8566-314cf9a9bd17\") " pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.248901 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.248901 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-serviceca\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.248901 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-cni-binary-copy\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248906    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxxjn\" (UniqueName: \"kubernetes.io/projected/fea4091a-9169-4cee-b245-2f909063cfb6-kube-api-access-hxxjn\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.248963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-system-cni-dir\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-slash\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-ovn\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-cni-netd\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-hostroot\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-etc-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wssk\" (UniqueName: \"kubernetes.io/projected/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-kube-api-access-9wssk\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-kubernetes\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-systemd\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-device-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-sys-fs\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/440d1a51-bd49-49b0-8566-314cf9a9bd17-konnectivity-ca\") pod \"konnectivity-agent-bnzhf\" (UID: \"440d1a51-bd49-49b0-8566-314cf9a9bd17\") " pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-systemd-units\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-systemd\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-host\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-host\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-cnibin\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cnibin\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-node-log\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-env-overrides\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-socket-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.249664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-registration-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-kubelet\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-etc-kubernetes\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-modprobe-d\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eab07679-827b-4064-b7d4-65c86f346a51-tmp\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-system-cni-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-cni-bin\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/086cc3c6-5330-4ef2-add4-e8892b85e117-kubelet-config\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-host-slash\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovnkube-script-lib\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysctl-conf\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-socket-dir-parent\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.249991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-netns\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c22c9\" (UniqueName: \"kubernetes.io/projected/8dd3f622-28c8-42d7-bd7c-db92655889ba-kube-api-access-c22c9\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-log-socket\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.250505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8sth\" (UniqueName: \"kubernetes.io/projected/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-kube-api-access-x8sth\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjpdm\" (UniqueName: \"kubernetes.io/projected/f174230a-a12c-4a6b-9f61-9c4aab306aa5-kube-api-access-mjpdm\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-lib-modules\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-cni-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/086cc3c6-5330-4ef2-add4-e8892b85e117-dbus\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-run-netns\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hf8g\" (UniqueName: \"kubernetes.io/projected/eab07679-827b-4064-b7d4-65c86f346a51-kube-api-access-8hf8g\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-multus-certs\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqrtj\" (UniqueName: \"kubernetes.io/projected/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-kube-api-access-zqrtj\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-sys\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-var-lib-kubelet\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkxtp\" (UniqueName: \"kubernetes.io/projected/673c4dae-4129-4359-8b99-b8e6e8723fe7-kube-api-access-pkxtp\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.251293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-k8s-cni-cncf-io\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.250976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-daemon-config\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fea4091a-9169-4cee-b245-2f909063cfb6-hosts-file\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-var-lib-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovnkube-config\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysconfig\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-cni-multus\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t68m\" (UniqueName: \"kubernetes.io/projected/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-kube-api-access-9t68m\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea4091a-9169-4cee-b245-2f909063cfb6-tmp-dir\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-iptables-alerter-script\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-run\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.251889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.251304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.309485 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.309457    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:27.337444 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.337417    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 23:01:27.348898 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.348870    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 23:01:27.352192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-system-cni-dir\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-slash\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352217    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-ovn\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-cni-netd\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-hostroot\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-system-cni-dir\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-ovn\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-etc-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-slash\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-hostroot\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wssk\" (UniqueName: \"kubernetes.io/projected/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-kube-api-access-9wssk\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-etc-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-cni-netd\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-kubernetes\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.352389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-systemd\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-device-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-sys-fs\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.352469    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/440d1a51-bd49-49b0-8566-314cf9a9bd17-konnectivity-ca\") pod \"konnectivity-agent-bnzhf\" (UID: \"440d1a51-bd49-49b0-8566-314cf9a9bd17\") " pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-systemd-units\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-kubernetes\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.352548    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:01:27.85251035 +0000 UTC m=+3.064481506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-systemd-units\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-systemd\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-systemd\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-host\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-host\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-systemd\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352692    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-sys-fs\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-device-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.353227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-host\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-cnibin\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-cnibin\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.352979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cnibin\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-node-log\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-host\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-env-overrides\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cnibin\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-socket-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-node-log\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-registration-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-kubelet\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-socket-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-etc-kubernetes\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-etc-kubernetes\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/440d1a51-bd49-49b0-8566-314cf9a9bd17-konnectivity-ca\") pod \"konnectivity-agent-bnzhf\" (UID: \"440d1a51-bd49-49b0-8566-314cf9a9bd17\") " pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-modprobe-d\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-registration-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.354044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-kubelet\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eab07679-827b-4064-b7d4-65c86f346a51-tmp\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-system-cni-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-cni-bin\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-modprobe-d\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/086cc3c6-5330-4ef2-add4-e8892b85e117-kubelet-config\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-host-slash\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-system-cni-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovnkube-script-lib\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-cni-bin\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysctl-conf\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-host-slash\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/086cc3c6-5330-4ef2-add4-e8892b85e117-kubelet-config\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-socket-dir-parent\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-env-overrides\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-netns\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.354971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c22c9\" (UniqueName: \"kubernetes.io/projected/8dd3f622-28c8-42d7-bd7c-db92655889ba-kube-api-access-c22c9\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-log-socket\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8sth\" (UniqueName: \"kubernetes.io/projected/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-kube-api-access-x8sth\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjpdm\" (UniqueName: \"kubernetes.io/projected/f174230a-a12c-4a6b-9f61-9c4aab306aa5-kube-api-access-mjpdm\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysctl-conf\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-lib-modules\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-cni-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-netns\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/086cc3c6-5330-4ef2-add4-e8892b85e117-dbus\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-log-socket\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-run-netns\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.353836    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.355820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-run-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.353897    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret podName:086cc3c6-5330-4ef2-add4-e8892b85e117 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:27.853879622 +0000 UTC m=+3.065850794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret") pod "global-pull-secret-syncer-j2kzp" (UID: "086cc3c6-5330-4ef2-add4-e8892b85e117") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353943    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hf8g\" (UniqueName: \"kubernetes.io/projected/eab07679-827b-4064-b7d4-65c86f346a51-kube-api-access-8hf8g\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.353719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-socket-dir-parent\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-multus-certs\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-cni-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-multus-certs\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovnkube-script-lib\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/086cc3c6-5330-4ef2-add4-e8892b85e117-dbus\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-run-netns\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-lib-modules\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqrtj\" (UniqueName: \"kubernetes.io/projected/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-kube-api-access-zqrtj\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-sys\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-var-lib-kubelet\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.356802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkxtp\" (UniqueName: \"kubernetes.io/projected/673c4dae-4129-4359-8b99-b8e6e8723fe7-kube-api-access-pkxtp\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-k8s-cni-cncf-io\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-var-lib-kubelet\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354413    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-sys\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-daemon-config\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fea4091a-9169-4cee-b245-2f909063cfb6-hosts-file\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-run-k8s-cni-cncf-io\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-var-lib-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fea4091a-9169-4cee-b245-2f909063cfb6-hosts-file\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovnkube-config\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysconfig\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-cni-multus\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t68m\" (UniqueName: \"kubernetes.io/projected/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-kube-api-access-9t68m\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea4091a-9169-4cee-b245-2f909063cfb6-tmp-dir\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-var-lib-openvswitch\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-host-var-lib-cni-multus\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.357716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-iptables-alerter-script\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-run\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354865    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysconfig\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-conf-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovn-node-metrics-cert\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.354999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysctl-d\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/673c4dae-4129-4359-8b99-b8e6e8723fe7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-daemon-config\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eab07679-827b-4064-b7d4-65c86f346a51-etc-tuned\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-kubelet\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-cni-bin\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/440d1a51-bd49-49b0-8566-314cf9a9bd17-agent-certs\") pod \"konnectivity-agent-bnzhf\" (UID: \"440d1a51-bd49-49b0-8566-314cf9a9bd17\") " pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-multus-conf-dir\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.358698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-serviceca\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-cni-binary-copy\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-run\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxxjn\" (UniqueName: \"kubernetes.io/projected/fea4091a-9169-4cee-b245-2f909063cfb6-kube-api-access-hxxjn\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovnkube-config\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f174230a-a12c-4a6b-9f61-9c4aab306aa5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-cni-bin\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fea4091a-9169-4cee-b245-2f909063cfb6-tmp-dir\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-iptables-alerter-script\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f174230a-a12c-4a6b-9f61-9c4aab306aa5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-serviceca\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355787    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eab07679-827b-4064-b7d4-65c86f346a51-etc-sysctl-d\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.355901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-host-kubelet\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.356144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-cni-binary-copy\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.358727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-ovn-node-metrics-cert\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.358852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/440d1a51-bd49-49b0-8566-314cf9a9bd17-agent-certs\") pod \"konnectivity-agent-bnzhf\" (UID: \"440d1a51-bd49-49b0-8566-314cf9a9bd17\") " pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.359457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.359160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eab07679-827b-4064-b7d4-65c86f346a51-etc-tuned\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.360337 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.359208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eab07679-827b-4064-b7d4-65c86f346a51-tmp\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.369794 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.369769    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:27.369906 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.369798    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:27.369906 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.369813    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pv524 for pod openshift-network-diagnostics/network-check-target-z7tt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:27.369906 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.369882    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524 podName:24b1faae-ba7c-4c7a-a647-05c0ff8504a9 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:27.86986383 +0000 UTC m=+3.081835003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pv524" (UniqueName: "kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524") pod "network-check-target-z7tt8" (UID: "24b1faae-ba7c-4c7a-a647-05c0ff8504a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:27.372206 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.372181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8sth\" (UniqueName: \"kubernetes.io/projected/dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5-kube-api-access-x8sth\") pod \"node-ca-h2f6t\" (UID: \"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5\") " pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.372301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.372201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wssk\" (UniqueName: \"kubernetes.io/projected/821b3ae3-4605-47ae-a13b-5027a4ac8b9b-kube-api-access-9wssk\") pod \"ovnkube-node-7mn4n\" (UID: \"821b3ae3-4605-47ae-a13b-5027a4ac8b9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.372301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.372217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c22c9\" (UniqueName: \"kubernetes.io/projected/8dd3f622-28c8-42d7-bd7c-db92655889ba-kube-api-access-c22c9\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:27.373156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.373128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxxjn\" (UniqueName: \"kubernetes.io/projected/fea4091a-9169-4cee-b245-2f909063cfb6-kube-api-access-hxxjn\") pod \"node-resolver-f9ttb\" (UID: \"fea4091a-9169-4cee-b245-2f909063cfb6\") " pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.373635 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.373594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjpdm\" (UniqueName: \"kubernetes.io/projected/f174230a-a12c-4a6b-9f61-9c4aab306aa5-kube-api-access-mjpdm\") pod \"multus-additional-cni-plugins-6hwws\" (UID: \"f174230a-a12c-4a6b-9f61-9c4aab306aa5\") " pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.374382 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.374355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t68m\" (UniqueName: \"kubernetes.io/projected/ab5aa01f-91a4-40dc-8c1e-a18e1697a45f-kube-api-access-9t68m\") pod \"multus-n8m6b\" (UID: \"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f\") " pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.375971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.375936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkxtp\" (UniqueName: \"kubernetes.io/projected/673c4dae-4129-4359-8b99-b8e6e8723fe7-kube-api-access-pkxtp\") pod \"aws-ebs-csi-driver-node-jdgfx\" (UID: \"673c4dae-4129-4359-8b99-b8e6e8723fe7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.376134 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.376106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hf8g\" (UniqueName: \"kubernetes.io/projected/eab07679-827b-4064-b7d4-65c86f346a51-kube-api-access-8hf8g\") pod \"tuned-dsf2b\" (UID: \"eab07679-827b-4064-b7d4-65c86f346a51\") " pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.376375 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.376359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqrtj\" (UniqueName: \"kubernetes.io/projected/4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c-kube-api-access-zqrtj\") pod \"iptables-alerter-qwbzn\" (UID: \"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c\") " pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.409574 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.409548    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c9vzn"
Sep 11 23:01:27.411844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.411795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal" event={"ID":"f3787794cb4c7a97e0f0378114cfc76b","Type":"ContainerStarted","Data":"ba34e6d871f890a512354cf712d1751cdeb8ac3ef8f8101113921e7f4ad90f5f"}
Sep 11 23:01:27.419614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.419591    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c9vzn"
Sep 11 23:01:27.499737 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.499662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h2f6t"
Sep 11 23:01:27.506820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.506792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dsf2b"
Sep 11 23:01:27.517913 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.517892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx"
Sep 11 23:01:27.524881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.524861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6hwws"
Sep 11 23:01:27.531751 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.531733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n8m6b"
Sep 11 23:01:27.538624 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.538602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:27.547421 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.547393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f9ttb"
Sep 11 23:01:27.556445 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.556421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qwbzn"
Sep 11 23:01:27.562338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.562319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:27.613152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.613130    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:27.743454 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.743417    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5aa01f_91a4_40dc_8c1e_a18e1697a45f.slice/crio-7b880bb8c08013e986e730f0cee7ba3bba02997d608ccf3da48f0918dc53782d WatchSource:0}: Error finding container 7b880bb8c08013e986e730f0cee7ba3bba02997d608ccf3da48f0918dc53782d: Status 404 returned error can't find the container with id 7b880bb8c08013e986e730f0cee7ba3bba02997d608ccf3da48f0918dc53782d
Sep 11 23:01:27.744188 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.744116    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab07679_827b_4064_b7d4_65c86f346a51.slice/crio-d83c271e266a4768d19fa377abc2c3005bc2847f62847f133afd8e975cc1d596 WatchSource:0}: Error finding container d83c271e266a4768d19fa377abc2c3005bc2847f62847f133afd8e975cc1d596: Status 404 returned error can't find the container with id d83c271e266a4768d19fa377abc2c3005bc2847f62847f133afd8e975cc1d596
Sep 11 23:01:27.748368 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.748317    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea4091a_9169_4cee_b245_2f909063cfb6.slice/crio-cc45d6564c60d975bfcf9843d83ca8c0b7b29cb9a91f03b50e1d4aa7afd5739e WatchSource:0}: Error finding container cc45d6564c60d975bfcf9843d83ca8c0b7b29cb9a91f03b50e1d4aa7afd5739e: Status 404 returned error can't find the container with id cc45d6564c60d975bfcf9843d83ca8c0b7b29cb9a91f03b50e1d4aa7afd5739e
Sep 11 23:01:27.749512 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.749485    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4593f8f4_7ce0_4cf7_a3c1_63f7a43fbd8c.slice/crio-add2bc4b770aa74af6759f607048eb47da8d37ee305646d7cf56dc3693ebc8e1 WatchSource:0}: Error finding container add2bc4b770aa74af6759f607048eb47da8d37ee305646d7cf56dc3693ebc8e1: Status 404 returned error can't find the container with id add2bc4b770aa74af6759f607048eb47da8d37ee305646d7cf56dc3693ebc8e1
Sep 11 23:01:27.750166 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.750096    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf174230a_a12c_4a6b_9f61_9c4aab306aa5.slice/crio-aee59985d4e7ec479a654efa626fcef59932bb6238e1ff14a117a3e38304b3ab WatchSource:0}: Error finding container aee59985d4e7ec479a654efa626fcef59932bb6238e1ff14a117a3e38304b3ab: Status 404 returned error can't find the container with id aee59985d4e7ec479a654efa626fcef59932bb6238e1ff14a117a3e38304b3ab
Sep 11 23:01:27.751562 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.751374    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673c4dae_4129_4359_8b99_b8e6e8723fe7.slice/crio-621bff4e9912aa81cb52c629fdb9d0637b9cdb03b0d39b5ec90a07feed98104d WatchSource:0}: Error finding container 621bff4e9912aa81cb52c629fdb9d0637b9cdb03b0d39b5ec90a07feed98104d: Status 404 returned error can't find the container with id 621bff4e9912aa81cb52c629fdb9d0637b9cdb03b0d39b5ec90a07feed98104d
Sep 11 23:01:27.755197 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.755171    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf7d1ff_9c3b_4cc5_ab5b_6d7ad6e965c5.slice/crio-a8d13f5c8e8ad1aa28bda733110b76b32dab2ae734c1cd350cbfab922ba3079f WatchSource:0}: Error finding container a8d13f5c8e8ad1aa28bda733110b76b32dab2ae734c1cd350cbfab922ba3079f: Status 404 returned error can't find the container with id a8d13f5c8e8ad1aa28bda733110b76b32dab2ae734c1cd350cbfab922ba3079f
Sep 11 23:01:27.860526 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.860490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:27.860647 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.860574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:27.860732 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.860648    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:27.860732 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.860721    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:27.860839 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.860725    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:01:28.860703961 +0000 UTC m=+4.072675129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:27.860839 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.860800    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret podName:086cc3c6-5330-4ef2-add4-e8892b85e117 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:28.860781786 +0000 UTC m=+4.072752949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret") pod "global-pull-secret-syncer-j2kzp" (UID: "086cc3c6-5330-4ef2-add4-e8892b85e117") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:27.913809 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:27.913782    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067c5e3fd43aceeec531c7141228c7cb.slice/crio-2a0d78237201c8fa2ec5a14b2b84d157f1899f5875baccf0a65e7a2bb1b63ede WatchSource:0}: Error finding container 2a0d78237201c8fa2ec5a14b2b84d157f1899f5875baccf0a65e7a2bb1b63ede: Status 404 returned error can't find the container with id 2a0d78237201c8fa2ec5a14b2b84d157f1899f5875baccf0a65e7a2bb1b63ede
Sep 11 23:01:27.960868 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:27.960842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:27.961004 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.960982    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:27.961058 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.961010    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:27.961058 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.961020    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pv524 for pod openshift-network-diagnostics/network-check-target-z7tt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:27.961124 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:27.961070    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524 podName:24b1faae-ba7c-4c7a-a647-05c0ff8504a9 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:28.961056344 +0000 UTC m=+4.173027498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pv524" (UniqueName: "kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524") pod "network-check-target-z7tt8" (UID: "24b1faae-ba7c-4c7a-a647-05c0ff8504a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:28.408476 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.408390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:28.409318 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.408553    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:28.425537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.423075    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 22:56:27 +0000 UTC" deadline="2028-05-31 20:00:03.778753802 +0000 UTC"
Sep 11 23:01:28.425537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.423112    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15068h58m35.355647093s"
Sep 11 23:01:28.425537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.424208    2580 generic.go:356] "Generic (PLEG): container finished" podID="f3787794cb4c7a97e0f0378114cfc76b" containerID="bd39e2381d11d8b44eb4d7f152de71e5f10571fe4b5b81820edc363ef89f832c" exitCode=0
Sep 11 23:01:28.425537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.424401    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal" event={"ID":"f3787794cb4c7a97e0f0378114cfc76b","Type":"ContainerDied","Data":"bd39e2381d11d8b44eb4d7f152de71e5f10571fe4b5b81820edc363ef89f832c"}
Sep 11 23:01:28.428088 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.428004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h2f6t" event={"ID":"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5","Type":"ContainerStarted","Data":"a8d13f5c8e8ad1aa28bda733110b76b32dab2ae734c1cd350cbfab922ba3079f"}
Sep 11 23:01:28.435602 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.435562    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bnzhf" event={"ID":"440d1a51-bd49-49b0-8566-314cf9a9bd17","Type":"ContainerStarted","Data":"4bcd157526822d7172958da492594edc9a04c1d003fd56255a92ab8a46432ed9"}
Sep 11 23:01:28.439389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.439365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"bb06a4a873aa3c38d847ec379c3351fe2ad49b36cd20f20ea6d2e9bbe49bc9aa"}
Sep 11 23:01:28.443857 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.443829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx" event={"ID":"673c4dae-4129-4359-8b99-b8e6e8723fe7","Type":"ContainerStarted","Data":"621bff4e9912aa81cb52c629fdb9d0637b9cdb03b0d39b5ec90a07feed98104d"}
Sep 11 23:01:28.450580 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.450551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal" event={"ID":"067c5e3fd43aceeec531c7141228c7cb","Type":"ContainerStarted","Data":"2a0d78237201c8fa2ec5a14b2b84d157f1899f5875baccf0a65e7a2bb1b63ede"}
Sep 11 23:01:28.453344 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.453317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerStarted","Data":"aee59985d4e7ec479a654efa626fcef59932bb6238e1ff14a117a3e38304b3ab"}
Sep 11 23:01:28.458173 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.458112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qwbzn" event={"ID":"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c","Type":"ContainerStarted","Data":"add2bc4b770aa74af6759f607048eb47da8d37ee305646d7cf56dc3693ebc8e1"}
Sep 11 23:01:28.461359 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.461307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f9ttb" event={"ID":"fea4091a-9169-4cee-b245-2f909063cfb6","Type":"ContainerStarted","Data":"cc45d6564c60d975bfcf9843d83ca8c0b7b29cb9a91f03b50e1d4aa7afd5739e"}
Sep 11 23:01:28.463492 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.463465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dsf2b" event={"ID":"eab07679-827b-4064-b7d4-65c86f346a51","Type":"ContainerStarted","Data":"d83c271e266a4768d19fa377abc2c3005bc2847f62847f133afd8e975cc1d596"}
Sep 11 23:01:28.468057 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.468030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n8m6b" event={"ID":"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f","Type":"ContainerStarted","Data":"7b880bb8c08013e986e730f0cee7ba3bba02997d608ccf3da48f0918dc53782d"}
Sep 11 23:01:28.870932 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.870890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:28.871147 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.871001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:28.871220 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.871169    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:28.871272 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.871239    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:01:30.871217252 +0000 UTC m=+6.083188406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:28.871763 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.871740    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:28.871893 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.871797    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret podName:086cc3c6-5330-4ef2-add4-e8892b85e117 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:30.871781277 +0000 UTC m=+6.083752431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret") pod "global-pull-secret-syncer-j2kzp" (UID: "086cc3c6-5330-4ef2-add4-e8892b85e117") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:28.972373 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:28.972277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:28.972543 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.972488    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:28.972543 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.972510    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:28.972543 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.972524    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pv524 for pod openshift-network-diagnostics/network-check-target-z7tt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:28.972722 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:28.972584    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524 podName:24b1faae-ba7c-4c7a-a647-05c0ff8504a9 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:30.97256622 +0000 UTC m=+6.184537379 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pv524" (UniqueName: "kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524") pod "network-check-target-z7tt8" (UID: "24b1faae-ba7c-4c7a-a647-05c0ff8504a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:29.408835 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:29.408613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:29.408835 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:29.408726    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:29.409391 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:29.408878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:29.409391 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:29.408955    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:29.423561 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:29.423461    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 22:56:27 +0000 UTC" deadline="2028-05-31 07:16:22.479859348 +0000 UTC"
Sep 11 23:01:29.423561 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:29.423497    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15056h14m53.056368535s"
Sep 11 23:01:29.486834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:29.486797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal" event={"ID":"f3787794cb4c7a97e0f0378114cfc76b","Type":"ContainerStarted","Data":"02c740c3668b63641c40666a2d39a8ac621451bcb57f41fa6a994b6ac52d0636"}
Sep 11 23:01:29.512330 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:29.512190    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-44.ec2.internal" podStartSLOduration=3.5121702089999998 podStartE2EDuration="3.512170209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:29.511966016 +0000 UTC m=+4.723937199" watchObservedRunningTime="2026-09-11 23:01:29.512170209 +0000 UTC m=+4.724141389"
Sep 11 23:01:30.408192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:30.408108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:30.408363 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.408260    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:30.891268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:30.891175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:30.891268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:30.891237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:30.891714 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.891355    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:30.891714 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.891416    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret podName:086cc3c6-5330-4ef2-add4-e8892b85e117 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:34.891401529 +0000 UTC m=+10.103372683 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret") pod "global-pull-secret-syncer-j2kzp" (UID: "086cc3c6-5330-4ef2-add4-e8892b85e117") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:30.891714 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.891355    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:30.891714 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.891527    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:01:34.891500028 +0000 UTC m=+10.103471195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:30.992284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:30.992250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:30.992450 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.992397    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:30.992450 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.992424    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:30.992450 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.992440    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pv524 for pod openshift-network-diagnostics/network-check-target-z7tt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:30.992626 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:30.992521    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524 podName:24b1faae-ba7c-4c7a-a647-05c0ff8504a9 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:34.99249931 +0000 UTC m=+10.204470478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pv524" (UniqueName: "kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524") pod "network-check-target-z7tt8" (UID: "24b1faae-ba7c-4c7a-a647-05c0ff8504a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:31.408103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:31.408071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:31.408280 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:31.408071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:31.408280 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:31.408225    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:31.408410 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:31.408290    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:32.407732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:32.407692    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:32.408113 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:32.407823    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:33.413178 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:33.408083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:33.413178 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:33.408219    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:33.413178 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:33.408265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:33.413178 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:33.408402    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:34.407702 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:34.407670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:34.407891 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:34.407810    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:34.922253 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:34.922214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:34.922704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:34.922297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:34.922704 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:34.922388    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:34.922704 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:34.922443    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:34.922704 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:34.922471    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:01:42.922449091 +0000 UTC m=+18.134420247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:34.922704 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:34.922503    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret podName:086cc3c6-5330-4ef2-add4-e8892b85e117 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:42.922487107 +0000 UTC m=+18.134458273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret") pod "global-pull-secret-syncer-j2kzp" (UID: "086cc3c6-5330-4ef2-add4-e8892b85e117") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:35.023752 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:35.023172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:35.023752 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:35.023353    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:35.023752 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:35.023374    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:35.023752 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:35.023388    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pv524 for pod openshift-network-diagnostics/network-check-target-z7tt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:35.023752 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:35.023440    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524 podName:24b1faae-ba7c-4c7a-a647-05c0ff8504a9 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:43.023423266 +0000 UTC m=+18.235394432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pv524" (UniqueName: "kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524") pod "network-check-target-z7tt8" (UID: "24b1faae-ba7c-4c7a-a647-05c0ff8504a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:35.411223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:35.410997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:35.411378 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:35.411316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:35.411631 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:35.411610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:35.411741 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:35.411716    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:35.501980 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:35.501881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bnzhf" event={"ID":"440d1a51-bd49-49b0-8566-314cf9a9bd17","Type":"ContainerStarted","Data":"cc27210c3eeb8a4d3d4501b8973cdf9be4feb6633648369647d73e638e17d099"}
Sep 11 23:01:35.504696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:35.504673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx" event={"ID":"673c4dae-4129-4359-8b99-b8e6e8723fe7","Type":"ContainerStarted","Data":"cb651b82761b1578da3e21238f4dd9f1db39dfc448b902cbe490b1b5589182ae"}
Sep 11 23:01:35.518268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:35.518206    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bnzhf" podStartSLOduration=2.248396962 podStartE2EDuration="9.518187946s" totalImagesPullingTime="7.269790984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:35.517545244 +0000 UTC m=+10.729516418" watchObservedRunningTime="2026-09-11 23:01:35.518187946 +0000 UTC m=+10.730159124"
Sep 11 23:01:36.408279 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:36.408251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:36.408721 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:36.408383    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:37.408129 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:37.408088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:37.408433 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:37.408213    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:37.408964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:37.408096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:37.408964 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:37.408627    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:38.408090 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:38.407999    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:38.408243 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:38.408154    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:38.804281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:38.804189    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:38.805142 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:38.805118    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:39.410932 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:39.410894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:39.411081 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:39.411029    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:39.411164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:39.411081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:39.411234 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:39.411205    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:39.512397 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:39.512360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:39.512889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:39.512872    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bnzhf"
Sep 11 23:01:40.407647 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:40.407622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:40.408093 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:40.407728    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:41.410617 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:41.410587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:41.411070 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:41.410712    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:41.411070 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:41.410765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:41.411070 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:41.410875    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:42.408535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:42.408504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:42.408737 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:42.408653    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:42.519022 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:42.518985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f9ttb" event={"ID":"fea4091a-9169-4cee-b245-2f909063cfb6","Type":"ContainerStarted","Data":"a5579fc5cc5d450dc2afcc20fd92ee5f8f0c771c6fac80fe2bc7e1559ac56fc2"}
Sep 11 23:01:42.533792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:42.533741    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f9ttb" podStartSLOduration=9.195605654 podStartE2EDuration="16.533719772s" totalImagesPullingTime="7.338114118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:42.532943301 +0000 UTC m=+17.744914477" watchObservedRunningTime="2026-09-11 23:01:42.533719772 +0000 UTC m=+17.745690945"
Sep 11 23:01:42.988396 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:42.988360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:42.988603 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:42.988428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:42.988603 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:42.988520    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:42.988603 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:42.988557    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:42.988603 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:42.988590    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:01:58.988571871 +0000 UTC m=+34.200543026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:42.988822 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:42.988618    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret podName:086cc3c6-5330-4ef2-add4-e8892b85e117 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:58.988607723 +0000 UTC m=+34.200578882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret") pod "global-pull-secret-syncer-j2kzp" (UID: "086cc3c6-5330-4ef2-add4-e8892b85e117") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:43.088806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:43.088772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:43.089008 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:43.088981    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:43.089008 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:43.089004    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:43.089136 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:43.089017    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pv524 for pod openshift-network-diagnostics/network-check-target-z7tt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:43.089136 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:43.089078    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524 podName:24b1faae-ba7c-4c7a-a647-05c0ff8504a9 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:59.089060165 +0000 UTC m=+34.301031332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pv524" (UniqueName: "kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524") pod "network-check-target-z7tt8" (UID: "24b1faae-ba7c-4c7a-a647-05c0ff8504a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:43.407726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:43.407642    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:43.407879 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:43.407796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:43.407957 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:43.407794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:43.407957 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:43.407924    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:44.408234 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:44.408198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:44.408780 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:44.408333    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:45.408479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:45.408452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:45.408951 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:45.408529    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:45.408951 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:45.408549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:45.408951 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:45.408649    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:46.408047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.408019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:46.408196 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:46.408172    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:46.530409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.529832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dsf2b" event={"ID":"eab07679-827b-4064-b7d4-65c86f346a51","Type":"ContainerStarted","Data":"847480ee370a6835c51cdf170cce1766de34abdfab1868ee9e1cf8ae5d55525c"}
Sep 11 23:01:46.531231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.531114    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 23:01:46.532720 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.532690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n8m6b" event={"ID":"ab5aa01f-91a4-40dc-8c1e-a18e1697a45f","Type":"ContainerStarted","Data":"26e4a33d157afb48c21ce7749f041bc7ca7921f5f38fed1e24d61a3e13f4e2b5"}
Sep 11 23:01:46.534300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.534274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h2f6t" event={"ID":"dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5","Type":"ContainerStarted","Data":"26131025dc847490b0b7376248274861e7c2cf74c4731f97d648c5551138f633"}
Sep 11 23:01:46.537318 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.537296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"6fd84266917636464b3880e498100901363a3b9acc75cf624272819036290da3"}
Sep 11 23:01:46.540047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.539842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx" event={"ID":"673c4dae-4129-4359-8b99-b8e6e8723fe7","Type":"ContainerStarted","Data":"af61d585da3f8bc3f0559725fb2c25b3cec070e64ea8f68c7ad6d8f2555569d1"}
Sep 11 23:01:46.541400 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.541379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal" event={"ID":"067c5e3fd43aceeec531c7141228c7cb","Type":"ContainerStarted","Data":"ec05781d0a51a684293cf09f75dfcf11232fb158c7bc5e726b7bf545977ad342"}
Sep 11 23:01:46.543144 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.543112    2580 generic.go:356] "Generic (PLEG): container finished" podID="f174230a-a12c-4a6b-9f61-9c4aab306aa5" containerID="985f354371e3ae06523329f8d94a909beb3c6a4ed4e5216d8e77a1af92ec2618" exitCode=0
Sep 11 23:01:46.543213 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.543151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerDied","Data":"985f354371e3ae06523329f8d94a909beb3c6a4ed4e5216d8e77a1af92ec2618"}
Sep 11 23:01:46.558871 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.558711    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dsf2b" podStartSLOduration=3.149305192 podStartE2EDuration="21.558679971s" totalImagesPullingTime="18.409374779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:46.558371829 +0000 UTC m=+21.770343007" watchObservedRunningTime="2026-09-11 23:01:46.558679971 +0000 UTC m=+21.770651155"
Sep 11 23:01:46.579730 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.579681    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-44.ec2.internal" podStartSLOduration=20.579658947 podStartE2EDuration="20.579658947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:46.579482138 +0000 UTC m=+21.791453313" watchObservedRunningTime="2026-09-11 23:01:46.579658947 +0000 UTC m=+21.791630122"
Sep 11 23:01:46.625676 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.625633    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n8m6b" podStartSLOduration=1.995441647 podStartE2EDuration="20.625615479s" totalImagesPullingTime="18.630173832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:46.625568359 +0000 UTC m=+21.837539533" watchObservedRunningTime="2026-09-11 23:01:46.625615479 +0000 UTC m=+21.837586654"
Sep 11 23:01:46.641924 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:46.641870    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h2f6t" podStartSLOduration=14.334303476 podStartE2EDuration="21.64185512s" totalImagesPullingTime="7.307551644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:46.641452407 +0000 UTC m=+21.853423583" watchObservedRunningTime="2026-09-11 23:01:46.64185512 +0000 UTC m=+21.853826294"
Sep 11 23:01:47.404514 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.404366    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T23:01:46.531127232Z","UUID":"e3d48a7f-7fd3-477d-8456-5e2df2d8bae2","Handler":null,"Name":"","Endpoint":""}
Sep 11 23:01:47.406259 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.406239    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 23:01:47.406387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.406272    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 23:01:47.407603 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.407583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:47.407731 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:47.407707    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:47.407799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.407751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:47.407851 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:47.407833    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:47.549783 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.549744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"dd3add4dfd582b3783753e3a1d8867eb2bd0ea464065f89da6dba21b7ae7537e"}
Sep 11 23:01:47.550295 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.549841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"54d63c479fc441e15d09ed689c8ee5df8c333a3de0badc5f4037f805186e7022"}
Sep 11 23:01:47.550295 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.549895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"5c209c2970187d277735346b41f939a5323f0e992a3409a81928373a3840b40d"}
Sep 11 23:01:47.550295 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.549964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"fb87af77c3858065f702eccd401bd07d364c7d6826556b82e997e476344b49df"}
Sep 11 23:01:47.550295 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.550017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"4cb46e079fe1d801d027b5e426b6821e1fe0a0cbc987eee9e4b4462fd7339c73"}
Sep 11 23:01:47.552304 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.552273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx" event={"ID":"673c4dae-4129-4359-8b99-b8e6e8723fe7","Type":"ContainerStarted","Data":"5da4fbaa2917a1604723a8788334f54c92a716c7a39fa5c2460adab738484e08"}
Sep 11 23:01:47.568883 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:47.568839    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdgfx" podStartSLOduration=2.347985656 podStartE2EDuration="21.568821901s" totalImagesPullingTime="19.220836245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:47.568573625 +0000 UTC m=+22.780544788" watchObservedRunningTime="2026-09-11 23:01:47.568821901 +0000 UTC m=+22.780793075"
Sep 11 23:01:48.407677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:48.407648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:48.407839 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:48.407753    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:48.556020 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:48.555983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qwbzn" event={"ID":"4593f8f4-7ce0-4cf7-a3c1-63f7a43fbd8c","Type":"ContainerStarted","Data":"baa20efaaea6d3c2babad3152790ec059c1f9ecaaffba737147f8e490885d282"}
Sep 11 23:01:49.411424 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:49.411390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:49.411606 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:49.411492    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:49.411606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:49.411573    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:49.411720 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:49.411674    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:49.562004 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:49.561968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"287e1ed2221110877625699611f3bfc547f17f05b7bd1f16ff4a752ddb20d11b"}
Sep 11 23:01:50.408291 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:50.408258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:50.408510 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:50.408398    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:51.410869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.410650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:51.411397 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.410699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:51.411397 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:51.410883    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:51.411397 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:51.410956    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:51.569216 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.569180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" event={"ID":"821b3ae3-4605-47ae-a13b-5027a4ac8b9b","Type":"ContainerStarted","Data":"3845b6c8aa5dcbcf4c53a1db5be863c67197ae4dea58620aff43e0292ecbf796"}
Sep 11 23:01:51.569424 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.569404    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:51.571225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.571196    2580 generic.go:356] "Generic (PLEG): container finished" podID="f174230a-a12c-4a6b-9f61-9c4aab306aa5" containerID="392eec0357012ffcd2364a7c4cf453a03331202f8bd0b42d3972b9f0732549a5" exitCode=0
Sep 11 23:01:51.571366 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.571239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerDied","Data":"392eec0357012ffcd2364a7c4cf453a03331202f8bd0b42d3972b9f0732549a5"}
Sep 11 23:01:51.584709 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.584684    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:51.595530 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.595484    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qwbzn" podStartSLOduration=7.204016678 podStartE2EDuration="25.595467838s" totalImagesPullingTime="18.39145116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:48.570623544 +0000 UTC m=+23.782594718" watchObservedRunningTime="2026-09-11 23:01:51.595467838 +0000 UTC m=+26.807439016"
Sep 11 23:01:51.595655 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:51.595634    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n" podStartSLOduration=6.9677613 podStartE2EDuration="25.595625199s" totalImagesPullingTime="18.627863899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:51.595438373 +0000 UTC m=+26.807409560" watchObservedRunningTime="2026-09-11 23:01:51.595625199 +0000 UTC m=+26.807596372"
Sep 11 23:01:52.408463 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.408436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:52.408572 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:52.408547    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:52.575781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.575704    2580 generic.go:356] "Generic (PLEG): container finished" podID="f174230a-a12c-4a6b-9f61-9c4aab306aa5" containerID="32e5f4226a7aee9ce308db6eb4f546ac45ec6aafb9eb9f8105bddd58de56bff6" exitCode=0
Sep 11 23:01:52.576346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.575788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerDied","Data":"32e5f4226a7aee9ce308db6eb4f546ac45ec6aafb9eb9f8105bddd58de56bff6"}
Sep 11 23:01:52.576346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.576320    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:52.576443 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.576407    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:52.593721 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.593694    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:01:52.846799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.846513    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j2kzp"]
Sep 11 23:01:52.846956 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.846866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:52.847023 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:52.847001    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:52.849710 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.849683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7tt8"]
Sep 11 23:01:52.849826 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.849794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:52.849929 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:52.849895    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:52.850295 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.850273    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9msgs"]
Sep 11 23:01:52.850388 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:52.850359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:52.850477 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:52.850446    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:53.580448 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:53.580357    2580 generic.go:356] "Generic (PLEG): container finished" podID="f174230a-a12c-4a6b-9f61-9c4aab306aa5" containerID="ce8f003625e75688cd33fc13d0dee92431c3542451bf857c2a30f1ab88c50aa6" exitCode=0
Sep 11 23:01:53.580448 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:53.580434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerDied","Data":"ce8f003625e75688cd33fc13d0dee92431c3542451bf857c2a30f1ab88c50aa6"}
Sep 11 23:01:54.408584 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:54.408553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:54.408766 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:54.408669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:54.408766 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:54.408679    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:54.408766 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:54.408699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:54.408941 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:54.408793    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:54.408941 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:54.408882    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:54.882928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:54.882829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:01:55.905482 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:55.905382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:01:56.018832 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:56.018738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:01:56.407689 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:56.407663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:56.407854 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:56.407663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:56.407854 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:56.407786    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:56.407854 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:56.407666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:56.407987 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:56.407862    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:56.407987 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:56.407928    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:56.673372 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:56.673283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:01:56.728284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:56.728248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:01:57.723070 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:57.723005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:01:57.822729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:57.822705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:01:58.408016 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:58.407984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:58.408283 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:58.407984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:58.408283 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:58.408083    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2kzp" podUID="086cc3c6-5330-4ef2-add4-e8892b85e117"
Sep 11 23:01:58.408283 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:58.407987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:58.408283 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:58.408154    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7tt8" podUID="24b1faae-ba7c-4c7a-a647-05c0ff8504a9"
Sep 11 23:01:58.408283 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:58.408268    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:01:58.594561 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:58.594524    2580 generic.go:356] "Generic (PLEG): container finished" podID="f174230a-a12c-4a6b-9f61-9c4aab306aa5" containerID="649b24d55ac2213813468d45e2aa617ddc9594f0e4102b27c3431f71cd6b3f35" exitCode=0
Sep 11 23:01:58.594727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:58.594592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerDied","Data":"649b24d55ac2213813468d45e2aa617ddc9594f0e4102b27c3431f71cd6b3f35"}
Sep 11 23:01:59.003881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.003667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:01:59.003881 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.003823    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:59.003881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.003857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:01:59.004303 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.003887    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:02:31.003870216 +0000 UTC m=+66.215841372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:59.004303 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.003958    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:59.004303 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.004004    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret podName:086cc3c6-5330-4ef2-add4-e8892b85e117 nodeName:}" failed. No retries permitted until 2026-09-11 23:02:31.003990689 +0000 UTC m=+66.215961841 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret") pod "global-pull-secret-syncer-j2kzp" (UID: "086cc3c6-5330-4ef2-add4-e8892b85e117") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:59.104806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.104773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:01:59.104983 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.104933    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:59.104983 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.104952    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:59.104983 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.104962    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pv524 for pod openshift-network-diagnostics/network-check-target-z7tt8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:59.105095 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.105012    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524 podName:24b1faae-ba7c-4c7a-a647-05c0ff8504a9 nodeName:}" failed. No retries permitted until 2026-09-11 23:02:31.104996727 +0000 UTC m=+66.316967879 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pv524" (UniqueName: "kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524") pod "network-check-target-z7tt8" (UID: "24b1faae-ba7c-4c7a-a647-05c0ff8504a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:59.157978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.157952    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-44.ec2.internal" event="NodeReady"
Sep 11 23:01:59.158100 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.158059    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 23:01:59.203766 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.203739    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-md4vn"]
Sep 11 23:01:59.228778 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.228754    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xrmq9"]
Sep 11 23:01:59.228948 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.228893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.231337 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.231315    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 23:01:59.231447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.231321    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 23:01:59.231580 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.231564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x84tr\""
Sep 11 23:01:59.252895 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.252874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-md4vn"]
Sep 11 23:01:59.253012 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.252907    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xrmq9"]
Sep 11 23:01:59.253012 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.252984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.256541 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.256457    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 23:01:59.256541 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.256493    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 23:01:59.256541 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.256515    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z767f\""
Sep 11 23:01:59.256833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.256662    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 23:01:59.306225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.306196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.306342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.306231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s8m8\" (UniqueName: \"kubernetes.io/projected/9a1cb00f-c928-4518-9867-cbec17193f4d-kube-api-access-6s8m8\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.306342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.306268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a1cb00f-c928-4518-9867-cbec17193f4d-tmp-dir\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.306342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.306293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1cb00f-c928-4518-9867-cbec17193f4d-config-volume\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.407529 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.407498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.407529 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.407530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s8m8\" (UniqueName: \"kubernetes.io/projected/9a1cb00f-c928-4518-9867-cbec17193f4d-kube-api-access-6s8m8\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.407750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.407552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25084dda-89f7-45d8-aced-38d4c22a2ce6-cert\") pod \"ingress-canary-xrmq9\" (UID: \"25084dda-89f7-45d8-aced-38d4c22a2ce6\") " pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.407750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.407588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a1cb00f-c928-4518-9867-cbec17193f4d-tmp-dir\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.407750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.407615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1cb00f-c928-4518-9867-cbec17193f4d-config-volume\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.407750 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.407649    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:01:59.407750 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.407713    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:01:59.907694454 +0000 UTC m=+35.119665621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:01:59.407750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.407734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqksp\" (UniqueName: \"kubernetes.io/projected/25084dda-89f7-45d8-aced-38d4c22a2ce6-kube-api-access-mqksp\") pod \"ingress-canary-xrmq9\" (UID: \"25084dda-89f7-45d8-aced-38d4c22a2ce6\") " pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.408074 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.407983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a1cb00f-c928-4518-9867-cbec17193f4d-tmp-dir\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.426107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.426076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s8m8\" (UniqueName: \"kubernetes.io/projected/9a1cb00f-c928-4518-9867-cbec17193f4d-kube-api-access-6s8m8\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.432511 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.432492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a1cb00f-c928-4518-9867-cbec17193f4d-config-volume\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.508897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.508815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25084dda-89f7-45d8-aced-38d4c22a2ce6-cert\") pod \"ingress-canary-xrmq9\" (UID: \"25084dda-89f7-45d8-aced-38d4c22a2ce6\") " pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.508897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.508885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqksp\" (UniqueName: \"kubernetes.io/projected/25084dda-89f7-45d8-aced-38d4c22a2ce6-kube-api-access-mqksp\") pod \"ingress-canary-xrmq9\" (UID: \"25084dda-89f7-45d8-aced-38d4c22a2ce6\") " pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.511388 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.511356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25084dda-89f7-45d8-aced-38d4c22a2ce6-cert\") pod \"ingress-canary-xrmq9\" (UID: \"25084dda-89f7-45d8-aced-38d4c22a2ce6\") " pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.516476 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.516455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqksp\" (UniqueName: \"kubernetes.io/projected/25084dda-89f7-45d8-aced-38d4c22a2ce6-kube-api-access-mqksp\") pod \"ingress-canary-xrmq9\" (UID: \"25084dda-89f7-45d8-aced-38d4c22a2ce6\") " pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.563575 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.563546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xrmq9"
Sep 11 23:01:59.600962 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.600928    2580 generic.go:356] "Generic (PLEG): container finished" podID="f174230a-a12c-4a6b-9f61-9c4aab306aa5" containerID="f52c93acb462f878f6e23390e1a17186060acd70bf8e7ba46a351f57653f7858" exitCode=0
Sep 11 23:01:59.601054 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.600945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerDied","Data":"f52c93acb462f878f6e23390e1a17186060acd70bf8e7ba46a351f57653f7858"}
Sep 11 23:01:59.879580 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.879497    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xrmq9"]
Sep 11 23:01:59.883708 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:01:59.883683    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25084dda_89f7_45d8_aced_38d4c22a2ce6.slice/crio-fd223d3450b174e19cb19b1a1def5f325fa570c8aa56c6d48e82d0e0a14eb742 WatchSource:0}: Error finding container fd223d3450b174e19cb19b1a1def5f325fa570c8aa56c6d48e82d0e0a14eb742: Status 404 returned error can't find the container with id fd223d3450b174e19cb19b1a1def5f325fa570c8aa56c6d48e82d0e0a14eb742
Sep 11 23:01:59.911640 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:01:59.911617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:01:59.911751 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.911733    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:01:59.911801 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:01:59.911792    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:02:00.911775091 +0000 UTC m=+36.123746242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:00.408357 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.408177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:02:00.409187 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.408200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:02:00.409187 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.408237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:02:00.411382 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.411352    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 23:02:00.411508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.411458    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 23:02:00.411570 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.411533    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 23:02:00.412371 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.412350    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 23:02:00.412371 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.412364    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rtlm9\""
Sep 11 23:02:00.412542 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.412416    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjwfh\""
Sep 11 23:02:00.608387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.608356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hwws" event={"ID":"f174230a-a12c-4a6b-9f61-9c4aab306aa5","Type":"ContainerStarted","Data":"fad4defac0706c57547f3c388626f919657ca2943de240bdcb851bcc2627684b"}
Sep 11 23:02:00.609712 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.609673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xrmq9" event={"ID":"25084dda-89f7-45d8-aced-38d4c22a2ce6","Type":"ContainerStarted","Data":"fd223d3450b174e19cb19b1a1def5f325fa570c8aa56c6d48e82d0e0a14eb742"}
Sep 11 23:02:00.631268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.631223    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6hwws" podStartSLOduration=17.929589694 podStartE2EDuration="34.631207719s" totalImagesPullingTime="16.701618025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:00.629677657 +0000 UTC m=+35.841648831" watchObservedRunningTime="2026-09-11 23:02:00.631207719 +0000 UTC m=+35.843178893"
Sep 11 23:02:00.918036 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:00.918000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:02:00.918194 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:00.918111    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:00.918194 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:00.918163    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:02:02.918149114 +0000 UTC m=+38.130120267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:01.572701 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:01.572668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:01.674137 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:01.674107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:02.933262 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:02.933178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:02:02.933686 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:02.933349    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:02.933686 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:02.933432    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:02:06.933412115 +0000 UTC m=+42.145383281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:03.618559 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:03.618524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xrmq9" event={"ID":"25084dda-89f7-45d8-aced-38d4c22a2ce6","Type":"ContainerStarted","Data":"b0f9059bc92f3bae2e2a12dfb5e76554e6b496742665b1d775747fd74e483a20"}
Sep 11 23:02:04.312756 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:04.312720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:04.369642 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:04.369611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:06.614241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:06.614213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:06.664461 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:06.664428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:06.958284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:06.958184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:02:06.958424 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:06.958295    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:06.958424 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:06.958357    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:02:14.958344277 +0000 UTC m=+50.170315428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:15.009091 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:15.009046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:02:15.009655 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:15.009179    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:15.009655 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:15.009265    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:02:31.0092436 +0000 UTC m=+66.221214775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:17.027108 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:17.027076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:17.092114 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:17.092078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:17.854551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:17.854482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:17.913310 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:17.913275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:24.593412 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:24.593386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7mn4n"
Sep 11 23:02:24.623168 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:24.623126    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xrmq9" podStartSLOduration=22.954043623 podStartE2EDuration="25.623111567s" totalImagesPullingTime="2.669067944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:03.633785696 +0000 UTC m=+38.845756869" watchObservedRunningTime="2026-09-11 23:02:24.623111567 +0000 UTC m=+59.835082742"
Sep 11 23:02:24.709535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:24.709511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:24.768260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:24.768239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:24.826791 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:24.826763    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:31.008141 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.008089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:02:31.008553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.008165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:02:31.011147 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.011121    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 23:02:31.011253 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.011121    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 23:02:31.018785 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:31.018767    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 23:02:31.018841 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:31.018832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:03:35.018816137 +0000 UTC m=+130.230787289 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : secret "metrics-daemon-secret" not found
Sep 11 23:02:31.021873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.021845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/086cc3c6-5330-4ef2-add4-e8892b85e117-original-pull-secret\") pod \"global-pull-secret-syncer-j2kzp\" (UID: \"086cc3c6-5330-4ef2-add4-e8892b85e117\") " pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:02:31.109046 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.109000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:02:31.109227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.109089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:02:31.109227 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:31.109182    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:31.109296 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:02:31.109239    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:03:03.109220925 +0000 UTC m=+98.321192094 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:31.112812 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.112793    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 23:02:31.122371 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.122351    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 23:02:31.132126 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.132101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv524\" (UniqueName: \"kubernetes.io/projected/24b1faae-ba7c-4c7a-a647-05c0ff8504a9-kube-api-access-pv524\") pod \"network-check-target-z7tt8\" (UID: \"24b1faae-ba7c-4c7a-a647-05c0ff8504a9\") " pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:02:31.321031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.321006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2kzp"
Sep 11 23:02:31.331195 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.331172    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjwfh\""
Sep 11 23:02:31.338983 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.338961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:02:31.492621 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.492561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:31.590237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.590174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:02:31.600289 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.600264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j2kzp"]
Sep 11 23:02:31.602934 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:02:31.602897    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086cc3c6_5330_4ef2_add4_e8892b85e117.slice/crio-bf8405dd229f4221fb14476d0fd5a6d0c21e75f0942f5ecf6c1adbec1af20545 WatchSource:0}: Error finding container bf8405dd229f4221fb14476d0fd5a6d0c21e75f0942f5ecf6c1adbec1af20545: Status 404 returned error can't find the container with id bf8405dd229f4221fb14476d0fd5a6d0c21e75f0942f5ecf6c1adbec1af20545
Sep 11 23:02:31.619753 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.619670    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7tt8"]
Sep 11 23:02:31.621883 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:02:31.621859    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b1faae_ba7c_4c7a_a647_05c0ff8504a9.slice/crio-e840261e48b8eda5d4a112bbc86ef5672c10e9276a9fb4b4fa5bd678101a2132 WatchSource:0}: Error finding container e840261e48b8eda5d4a112bbc86ef5672c10e9276a9fb4b4fa5bd678101a2132: Status 404 returned error can't find the container with id e840261e48b8eda5d4a112bbc86ef5672c10e9276a9fb4b4fa5bd678101a2132
Sep 11 23:02:31.687183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.687149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j2kzp" event={"ID":"086cc3c6-5330-4ef2-add4-e8892b85e117","Type":"ContainerStarted","Data":"bf8405dd229f4221fb14476d0fd5a6d0c21e75f0942f5ecf6c1adbec1af20545"}
Sep 11 23:02:31.688212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:31.688191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7tt8" event={"ID":"24b1faae-ba7c-4c7a-a647-05c0ff8504a9","Type":"ContainerStarted","Data":"e840261e48b8eda5d4a112bbc86ef5672c10e9276a9fb4b4fa5bd678101a2132"}
Sep 11 23:02:36.703659 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:36.703626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j2kzp" event={"ID":"086cc3c6-5330-4ef2-add4-e8892b85e117","Type":"ContainerStarted","Data":"ea3e30b3e2d064290f60bcf78f33e870f55f591ea6b64c97003916433cfcbc91"}
Sep 11 23:02:36.705244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:36.705210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7tt8" event={"ID":"24b1faae-ba7c-4c7a-a647-05c0ff8504a9","Type":"ContainerStarted","Data":"53911c358c8c988482421e247f2df437b0ceb5f9a13467e486a252bcc5678ffd"}
Sep 11 23:02:36.705373 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:36.705360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:02:36.718889 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:36.718841    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j2kzp" podStartSLOduration=65.764847789 podStartE2EDuration="1m10.718830391s" totalImagesPullingTime="4.953982602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:36.718582992 +0000 UTC m=+71.930554166" watchObservedRunningTime="2026-09-11 23:02:36.718830391 +0000 UTC m=+71.930801564"
Sep 11 23:02:36.733578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:36.733540    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z7tt8" podStartSLOduration=65.801866303 podStartE2EDuration="1m10.733527209s" totalImagesPullingTime="4.931660906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:36.733015629 +0000 UTC m=+71.944986805" watchObservedRunningTime="2026-09-11 23:02:36.733527209 +0000 UTC m=+71.945498382"
Sep 11 23:02:48.980992 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.980869    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"]
Sep 11 23:02:48.984613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.984593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:48.987751 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.987729    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"]
Sep 11 23:02:48.988072 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.988053    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kjpjs\""
Sep 11 23:02:48.988150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.988074    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 23:02:48.988209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.988158    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 23:02:48.988209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.988196    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 23:02:48.988437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.988421    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 23:02:48.990690 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.990666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:48.995223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.995208    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 23:02:48.995725 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.995711    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 23:02:48.995786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.995772    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 23:02:48.995820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:48.995788    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 23:02:49.000315 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.000297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"]
Sep 11 23:02:49.002762 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.002740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"]
Sep 11 23:02:49.026840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.026818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2fe4cadc-339a-48a0-a558-a96ee8a8dd1d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b6558f688-9vpkp\" (UID: \"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:49.026944 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.026858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnjmd\" (UniqueName: \"kubernetes.io/projected/2fe4cadc-339a-48a0-a558-a96ee8a8dd1d-kube-api-access-xnjmd\") pod \"managed-serviceaccount-addon-agent-b6558f688-9vpkp\" (UID: \"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:49.026944 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.026880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfw9m\" (UniqueName: \"kubernetes.io/projected/7b256dd2-45b9-4708-97f8-f9ff82fea416-kube-api-access-gfw9m\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.026944 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.026928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b256dd2-45b9-4708-97f8-f9ff82fea416-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.027051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.026966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.027051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.026989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.027051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.027030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-ca\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.027051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.027045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-hub\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.127957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.127933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.128103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.127975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-ca\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.128103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.127999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-hub\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.128103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.128032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2fe4cadc-339a-48a0-a558-a96ee8a8dd1d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b6558f688-9vpkp\" (UID: \"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:49.128103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.128071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnjmd\" (UniqueName: \"kubernetes.io/projected/2fe4cadc-339a-48a0-a558-a96ee8a8dd1d-kube-api-access-xnjmd\") pod \"managed-serviceaccount-addon-agent-b6558f688-9vpkp\" (UID: \"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:49.128301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.128105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfw9m\" (UniqueName: \"kubernetes.io/projected/7b256dd2-45b9-4708-97f8-f9ff82fea416-kube-api-access-gfw9m\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.128301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.128150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b256dd2-45b9-4708-97f8-f9ff82fea416-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.128301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.128178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.128930 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.128887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b256dd2-45b9-4708-97f8-f9ff82fea416-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.130648 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.130620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-ca\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.130743 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.130720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-hub\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.130839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.130825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2fe4cadc-339a-48a0-a558-a96ee8a8dd1d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b6558f688-9vpkp\" (UID: \"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:49.130874 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.130851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.131044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.131026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b256dd2-45b9-4708-97f8-f9ff82fea416-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.136422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.136395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnjmd\" (UniqueName: \"kubernetes.io/projected/2fe4cadc-339a-48a0-a558-a96ee8a8dd1d-kube-api-access-xnjmd\") pod \"managed-serviceaccount-addon-agent-b6558f688-9vpkp\" (UID: \"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:49.136961 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.136939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfw9m\" (UniqueName: \"kubernetes.io/projected/7b256dd2-45b9-4708-97f8-f9ff82fea416-kube-api-access-gfw9m\") pod \"cluster-proxy-proxy-agent-5647d8c496-l4lmz\" (UID: \"7b256dd2-45b9-4708-97f8-f9ff82fea416\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.303824 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.303751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"
Sep 11 23:02:49.310745 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.310693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:02:49.577907 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.577880    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp"]
Sep 11 23:02:49.580668 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:02:49.580639    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fe4cadc_339a_48a0_a558_a96ee8a8dd1d.slice/crio-f02f54db8b565c550ac2d9431a9d0e94882194c43bdcae48f74f477267c65c6c WatchSource:0}: Error finding container f02f54db8b565c550ac2d9431a9d0e94882194c43bdcae48f74f477267c65c6c: Status 404 returned error can't find the container with id f02f54db8b565c550ac2d9431a9d0e94882194c43bdcae48f74f477267c65c6c
Sep 11 23:02:49.584994 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.584970    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"]
Sep 11 23:02:49.588623 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:02:49.588601    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b256dd2_45b9_4708_97f8_f9ff82fea416.slice/crio-60a38b443d5037da2047e7f163636ffffe24172af053e0f362f55ddcb10c7a47 WatchSource:0}: Error finding container 60a38b443d5037da2047e7f163636ffffe24172af053e0f362f55ddcb10c7a47: Status 404 returned error can't find the container with id 60a38b443d5037da2047e7f163636ffffe24172af053e0f362f55ddcb10c7a47
Sep 11 23:02:49.736497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.736463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp" event={"ID":"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d","Type":"ContainerStarted","Data":"f02f54db8b565c550ac2d9431a9d0e94882194c43bdcae48f74f477267c65c6c"}
Sep 11 23:02:49.737575 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:49.737545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" event={"ID":"7b256dd2-45b9-4708-97f8-f9ff82fea416","Type":"ContainerStarted","Data":"60a38b443d5037da2047e7f163636ffffe24172af053e0f362f55ddcb10c7a47"}
Sep 11 23:02:53.752405 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:53.752373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp" event={"ID":"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d","Type":"ContainerStarted","Data":"7cb303422897ca7898c349a39ca11edf2fb6d68674582515bcffe14fdc85a99e"}
Sep 11 23:02:53.754495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:53.754470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" event={"ID":"7b256dd2-45b9-4708-97f8-f9ff82fea416","Type":"ContainerStarted","Data":"44915cc7aa19b7ba56a7fbb50224bbe40671dcf937be2941f4ec5fcbc0a763ea"}
Sep 11 23:02:53.754596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:53.754506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" event={"ID":"7b256dd2-45b9-4708-97f8-f9ff82fea416","Type":"ContainerStarted","Data":"b3439a27257719bf7cdd9abbda70b4a71692fd72371337517f131b934820c0f6"}
Sep 11 23:02:53.754596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:53.754523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" event={"ID":"7b256dd2-45b9-4708-97f8-f9ff82fea416","Type":"ContainerStarted","Data":"09980d3b40fbb8d7631bc597864a9ad902776e07cb8416e1f709367ac8fa949e"}
Sep 11 23:02:53.768884 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:53.768845    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp" podStartSLOduration=1.7928565459999999 podStartE2EDuration="5.768830802s" totalImagesPullingTime="3.975974256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:53.768337263 +0000 UTC m=+88.980308440" watchObservedRunningTime="2026-09-11 23:02:53.768830802 +0000 UTC m=+88.980801976"
Sep 11 23:02:53.790537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:02:53.790496    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" podStartSLOduration=1.811062523 podStartE2EDuration="5.790480675s" totalImagesPullingTime="3.979418152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:53.789747223 +0000 UTC m=+89.001718398" watchObservedRunningTime="2026-09-11 23:02:53.790480675 +0000 UTC m=+89.002451849"
Sep 11 23:03:01.838728 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:03:01.838692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:03:03.127370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:03:03.127317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:03:03.127735 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:03:03.127466    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:03:03.127735 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:03:03.127544    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:04:07.127527929 +0000 UTC m=+162.339499080 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:03:07.709691 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:03:07.709660    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z7tt8"
Sep 11 23:03:35.048633 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:03:35.048576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:03:35.049134 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:03:35.048732    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 23:03:35.049134 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:03:35.048798    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:05:37.048783092 +0000 UTC m=+252.260754245 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : secret "metrics-daemon-secret" not found
Sep 11 23:04:02.237500 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:04:02.237453    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-md4vn" podUID="9a1cb00f-c928-4518-9867-cbec17193f4d"
Sep 11 23:04:02.949542 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:04:02.949505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-md4vn"
Sep 11 23:04:03.438346 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:04:03.438291    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:04:07.160470 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:04:07.160418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:04:07.160890 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:04:07.160561    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:04:07.160890 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:04:07.160628    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls podName:9a1cb00f-c928-4518-9867-cbec17193f4d nodeName:}" failed. No retries permitted until 2026-09-11 23:06:09.16061264 +0000 UTC m=+284.372583795 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls") pod "dns-default-md4vn" (UID: "9a1cb00f-c928-4518-9867-cbec17193f4d") : secret "dns-default-metrics-tls" not found
Sep 11 23:04:14.408520 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:04:14.408482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:04:54.092942 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:04:54.092891    2580 generic.go:356] "Generic (PLEG): container finished" podID="2fe4cadc-339a-48a0-a558-a96ee8a8dd1d" containerID="7cb303422897ca7898c349a39ca11edf2fb6d68674582515bcffe14fdc85a99e" exitCode=255
Sep 11 23:04:54.093329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:04:54.092967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp" event={"ID":"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d","Type":"ContainerDied","Data":"7cb303422897ca7898c349a39ca11edf2fb6d68674582515bcffe14fdc85a99e"}
Sep 11 23:04:54.093329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:04:54.093290    2580 scope.go:117] "RemoveContainer" containerID="7cb303422897ca7898c349a39ca11edf2fb6d68674582515bcffe14fdc85a99e"
Sep 11 23:04:55.097596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:04:55.097557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b6558f688-9vpkp" event={"ID":"2fe4cadc-339a-48a0-a558-a96ee8a8dd1d","Type":"ContainerStarted","Data":"302d51b3a38c47a4288771f006bf0ea00519fcd3d6197f505266b93d6803ed53"}
Sep 11 23:05:37.144517 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:37.144462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:05:37.145026 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:37.144585    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 23:05:37.145026 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:37.144640    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs podName:8dd3f622-28c8-42d7-bd7c-db92655889ba nodeName:}" failed. No retries permitted until 2026-09-11 23:07:39.144624595 +0000 UTC m=+374.356595746 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs") pod "network-metrics-daemon-9msgs" (UID: "8dd3f622-28c8-42d7-bd7c-db92655889ba") : secret "metrics-daemon-secret" not found
Sep 11 23:05:42.285119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:42.285070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:42.342805 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:42.342775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:44.402737 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:44.402702    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:44.461559 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:44.461530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:46.251062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.251030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:46.314033 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.314006    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:46.506040 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.505975    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vs8dz"]
Sep 11 23:05:46.508999 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.508982    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.513438 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.513415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 23:05:46.513553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.513416    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 23:05:46.514282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.514265    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 23:05:46.514382 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.514306    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 23:05:46.514579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.514564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-thjf7\""
Sep 11 23:05:46.543103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.543076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:46.548881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.548854    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vs8dz"]
Sep 11 23:05:46.608072 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.608046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd3ca046-9348-4291-a9a9-4ce870b83f00-crio-socket\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.608176 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.608079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.608216 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.608175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.608216 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.608199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgzxn\" (UniqueName: \"kubernetes.io/projected/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-api-access-mgzxn\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.608282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.608223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd3ca046-9348-4291-a9a9-4ce870b83f00-data-volume\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.709512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.709471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.709638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.709518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgzxn\" (UniqueName: \"kubernetes.io/projected/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-api-access-mgzxn\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.709638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.709555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd3ca046-9348-4291-a9a9-4ce870b83f00-data-volume\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.709638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.709578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd3ca046-9348-4291-a9a9-4ce870b83f00-crio-socket\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.709638 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:46.709584    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:46.709638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.709604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.709864 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:46.709658    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:47.209637821 +0000 UTC m=+262.421609014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:46.709864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.709700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd3ca046-9348-4291-a9a9-4ce870b83f00-crio-socket\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.709864 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:46.709714    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:46.709864 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:46.709782    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:47.209765536 +0000 UTC m=+262.421736704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:46.709864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.709852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd3ca046-9348-4291-a9a9-4ce870b83f00-data-volume\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.717118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.717092    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:46.718592 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.718569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgzxn\" (UniqueName: \"kubernetes.io/projected/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-api-access-mgzxn\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:46.818714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.818693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:46.918613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:46.918585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:47.023325 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:47.023299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:47.214241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:47.214163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:47.214241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:47.214207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:47.214411 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:47.214287    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:47.214411 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:47.214334    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:47.214411 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:47.214353    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:48.214334397 +0000 UTC m=+263.426305563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:47.214411 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:47.214377    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:48.214366129 +0000 UTC m=+263.426337284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:47.909276 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:47.909241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:48.006393 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:48.006365    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:48.203778 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:48.203711    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f9ttb_fea4091a-9169-4cee-b245-2f909063cfb6/dns-node-resolver/0.log"
Sep 11 23:05:48.221031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:48.221000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:48.221155 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:48.221058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:48.221155 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:48.221119    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:48.221155 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:48.221141    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:48.221274 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:48.221181    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:50.221161408 +0000 UTC m=+265.433132567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:48.221274 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:48.221199    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:50.221187507 +0000 UTC m=+265.433158658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:48.598074 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:48.598042    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:48.663822 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:48.663795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:49.207793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:49.207768    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h2f6t_dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5/node-ca/0.log"
Sep 11 23:05:49.637758 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:49.637722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:50.233781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:50.233750    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:50.234189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:50.233810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:50.234189 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:50.233900    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:50.234189 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:50.233974    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:54.233959329 +0000 UTC m=+269.445930482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:50.234189 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:50.233900    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:50.234189 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:50.234060    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:54.234042905 +0000 UTC m=+269.446014057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:50.404301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:50.404271    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xrmq9_25084dda-89f7-45d8-aced-38d4c22a2ce6/serve-healthcheck-canary/0.log"
Sep 11 23:05:51.685835 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:51.685795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:51.816798 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:51.816749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:51.916885 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:51.916851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:52.030732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:52.030598    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:52.158678 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:52.158638    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:52.362158 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:52.362121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:52.505492 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:52.505455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:52.605767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:52.605738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:53.192972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:53.192938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:53.598927 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:53.598887    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:53.989208 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:53.989123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:54.262791 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:54.262698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:54.263237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:54.262791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:05:54.263237 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:54.262856    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:54.263237 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:54.262880    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:54.263237 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:54.262956    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:06:02.262938 +0000 UTC m=+277.474909170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:54.263237 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:05:54.262974    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm podName:bd3ca046-9348-4291-a9a9-4ce870b83f00 nodeName:}" failed. No retries permitted until 2026-09-11 23:06:02.26296545 +0000 UTC m=+277.474936601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vs8dz" (UID: "bd3ca046-9348-4291-a9a9-4ce870b83f00") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:54.386848 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:54.386813    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:54.605139 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:54.605110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:54.987008 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:54.986903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:55.382030 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:55.381992    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:55.601872 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:55.601791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:55.771868 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:55.771834    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:56.182194 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:56.182161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:56.384586 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:56.384544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:56.588162 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:56.588118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:56.787324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:56.787296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:57.782569 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:57.782536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:58.175824 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:58.175786    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:58.271382 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:58.271333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:58.385656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:58.385627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:58.583388 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:58.583359    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:58.781260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:58.781229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:59.736930 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:59.736883    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:05:59.988477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:05:59.988400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:00.187201 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:00.187168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:00.389026 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:00.388987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:00.648959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:00.648859    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:00.779904 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:00.779867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:00.924214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:00.924136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:01.595457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:01.595426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:01.786991 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:01.786936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:02.180704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.180673    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:02.325757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.325714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:06:02.325958 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.325822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:06:02.326357 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.326334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd3ca046-9348-4291-a9a9-4ce870b83f00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:06:02.328004 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.327985    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd3ca046-9348-4291-a9a9-4ce870b83f00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vs8dz\" (UID: \"bd3ca046-9348-4291-a9a9-4ce870b83f00\") " pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:06:02.387762 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.387729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:02.417852 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.417811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vs8dz"
Sep 11 23:06:02.688472 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:02.688438    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vs8dz"]
Sep 11 23:06:02.692879 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:06:02.692854    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3ca046_9348_4291_a9a9_4ce870b83f00.slice/crio-a652ed4850fd35e68eee00207cc9b0e7f5273876b37afc2d485f211a0328bbad WatchSource:0}: Error finding container a652ed4850fd35e68eee00207cc9b0e7f5273876b37afc2d485f211a0328bbad: Status 404 returned error can't find the container with id a652ed4850fd35e68eee00207cc9b0e7f5273876b37afc2d485f211a0328bbad
Sep 11 23:06:03.293152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:03.293110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs8dz" event={"ID":"bd3ca046-9348-4291-a9a9-4ce870b83f00","Type":"ContainerStarted","Data":"c7043a1015f3d91f3772ac02468ac649813d41c819ad0a3c070f61d0074554b5"}
Sep 11 23:06:03.293620 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:03.293197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs8dz" event={"ID":"bd3ca046-9348-4291-a9a9-4ce870b83f00","Type":"ContainerStarted","Data":"a652ed4850fd35e68eee00207cc9b0e7f5273876b37afc2d485f211a0328bbad"}
Sep 11 23:06:04.299048 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:04.299012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs8dz" event={"ID":"bd3ca046-9348-4291-a9a9-4ce870b83f00","Type":"ContainerStarted","Data":"f7aeacd050d925e7b03a2bb7fcc55e1bcf0fd84d16f8df3d75613b895b5c9d17"}
Sep 11 23:06:05.304473 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:05.304435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vs8dz" event={"ID":"bd3ca046-9348-4291-a9a9-4ce870b83f00","Type":"ContainerStarted","Data":"e47b94dc0baa5dfdb6d78956a26d9af9f3bad3285e124a7e56332fd856da2651"}
Sep 11 23:06:05.324363 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:05.324316    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vs8dz" podStartSLOduration=16.994764316 podStartE2EDuration="19.324301548s" totalImagesPullingTime="2.329537232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:05:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:05.322680404 +0000 UTC m=+280.534651579" watchObservedRunningTime="2026-09-11 23:06:05.324301548 +0000 UTC m=+280.536272721"
Sep 11 23:06:05.950272 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:06:05.950221    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-md4vn" podUID="9a1cb00f-c928-4518-9867-cbec17193f4d"
Sep 11 23:06:06.307596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:06.307517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-md4vn"
Sep 11 23:06:06.601942 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:06.601897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:06.658679 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:06.658648    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:07.700715 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:07.700685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:07.801467 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:07.801435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:08.552368 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:08.552339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:08.716740 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:08.716701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:08.901491 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:08.901456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:09.038541 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.038511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:09.174660 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.174590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:06:09.176843 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.176814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a1cb00f-c928-4518-9867-cbec17193f4d-metrics-tls\") pod \"dns-default-md4vn\" (UID: \"9a1cb00f-c928-4518-9867-cbec17193f4d\") " pod="openshift-dns/dns-default-md4vn"
Sep 11 23:06:09.207409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.207382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:09.278222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.278189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:09.310493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.310470    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x84tr\""
Sep 11 23:06:09.318469 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.318452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-md4vn"
Sep 11 23:06:09.592232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.592202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-md4vn"]
Sep 11 23:06:09.595682 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:06:09.595655    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1cb00f_c928_4518_9867_cbec17193f4d.slice/crio-65283699eb37c0bc50a3363c999581d696300869936af962cf2127dd6fed9520 WatchSource:0}: Error finding container 65283699eb37c0bc50a3363c999581d696300869936af962cf2127dd6fed9520: Status 404 returned error can't find the container with id 65283699eb37c0bc50a3363c999581d696300869936af962cf2127dd6fed9520
Sep 11 23:06:09.598133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.598104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:09.675237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:09.675207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:10.320972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:10.320932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-md4vn" event={"ID":"9a1cb00f-c928-4518-9867-cbec17193f4d","Type":"ContainerStarted","Data":"65283699eb37c0bc50a3363c999581d696300869936af962cf2127dd6fed9520"}
Sep 11 23:06:10.406757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:10.406724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:10.610143 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:10.610050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:10.823328 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:10.823295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:10.885711 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:10.885630    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:11.325574 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:11.325537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-md4vn" event={"ID":"9a1cb00f-c928-4518-9867-cbec17193f4d","Type":"ContainerStarted","Data":"687a5d63e56b07dd3fda9e537090907752fff37f99eb06470b7f2223a6aea7fb"}
Sep 11 23:06:11.379278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:11.379247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:12.330148 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:12.330113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-md4vn" event={"ID":"9a1cb00f-c928-4518-9867-cbec17193f4d","Type":"ContainerStarted","Data":"5afb0d2eb07eab7ac9278e8999e265a42a4d5f786b24ed18015fccad97d0d23a"}
Sep 11 23:06:12.330619 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:12.330235    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-md4vn"
Sep 11 23:06:12.347612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:12.347572    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-md4vn" podStartSLOduration=251.771909399 podStartE2EDuration="4m13.347558445s" totalImagesPullingTime="1.575649046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:12.346281916 +0000 UTC m=+287.558253090" watchObservedRunningTime="2026-09-11 23:06:12.347558445 +0000 UTC m=+287.559529618"
Sep 11 23:06:16.336238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:16.336199    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:16.493545 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:16.493505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:16.610298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:16.610200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:17.409371 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:06:17.409331    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9msgs" podUID="8dd3f622-28c8-42d7-bd7c-db92655889ba"
Sep 11 23:06:17.433948 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:17.433894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:17.530281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:17.530254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:19.626053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:19.626007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:20.939890 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:20.939854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:21.603387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:21.603356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:21.661655 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:21.661624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:22.335708 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:22.335683    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-md4vn"
Sep 11 23:06:23.513346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:23.513317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:23.569467 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:23.569441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:23.627289 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:23.627265    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:23.737906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:23.737880    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:23.798049 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:23.797978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:25.352690 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:25.352568    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 23:06:26.226688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.226655    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:26.281842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.281809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:26.425519 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.425484    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-755d6987f9-x4sck"]
Sep 11 23:06:26.428358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.428342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.432046 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 23:06:26.432297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432088    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 23:06:26.432297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432104    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 23:06:26.432297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432117    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 23:06:26.432297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432118    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vsph4\""
Sep 11 23:06:26.432297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432224    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 23:06:26.432479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432444    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 23:06:26.432514 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.432481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 23:06:26.439845 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.439756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-755d6987f9-x4sck"]
Sep 11 23:06:26.450864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.450832    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 23:06:26.492204 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.492106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-serving-cert\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.492204 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.492167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-oauth-config\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.492422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.492212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-oauth-serving-cert\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.492422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.492237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-service-ca\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.492422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.492314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tgpj\" (UniqueName: \"kubernetes.io/projected/217d8208-48c8-4cd5-a19e-72fbdd288440-kube-api-access-9tgpj\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.492422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.492391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-console-config\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.492550 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.492430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-trusted-ca-bundle\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.528826 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.528786    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:26.593223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.593190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-oauth-serving-cert\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.593223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.593226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-service-ca\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.593431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.593263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tgpj\" (UniqueName: \"kubernetes.io/projected/217d8208-48c8-4cd5-a19e-72fbdd288440-kube-api-access-9tgpj\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.593431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.593295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-console-config\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.593431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.593312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-trusted-ca-bundle\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.593431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.593342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-serving-cert\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.593668 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.593643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-oauth-config\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.594077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.594048    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-oauth-serving-cert\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.594200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.594048    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-service-ca\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.594200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.594182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-console-config\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.594796 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.594759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-trusted-ca-bundle\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.595029 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.595010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:26.596325 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.596300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-serving-cert\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.596325 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.596321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-oauth-config\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.601018 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.600994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tgpj\" (UniqueName: \"kubernetes.io/projected/217d8208-48c8-4cd5-a19e-72fbdd288440-kube-api-access-9tgpj\") pod \"console-755d6987f9-x4sck\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") " pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:26.738433 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:26.738388    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:27.010602 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:27.010527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-755d6987f9-x4sck"]
Sep 11 23:06:27.015647 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:06:27.015605    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217d8208_48c8_4cd5_a19e_72fbdd288440.slice/crio-d29090ba86f5cb725fbda3a09da3a373393ca885d22faff1f4ea8583385fee86 WatchSource:0}: Error finding container d29090ba86f5cb725fbda3a09da3a373393ca885d22faff1f4ea8583385fee86: Status 404 returned error can't find the container with id d29090ba86f5cb725fbda3a09da3a373393ca885d22faff1f4ea8583385fee86
Sep 11 23:06:27.018429 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:27.018414    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:06:27.165751 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:27.165707    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:27.278178 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:27.278109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:27.377801 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:27.377746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:27.377977 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:27.377826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-755d6987f9-x4sck" event={"ID":"217d8208-48c8-4cd5-a19e-72fbdd288440","Type":"ContainerStarted","Data":"d29090ba86f5cb725fbda3a09da3a373393ca885d22faff1f4ea8583385fee86"}
Sep 11 23:06:27.482319 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:27.482262    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:29.709668 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:29.709627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:29.816261 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:29.816231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:29.918181 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:29.918085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:30.225322 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:30.225219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:30.326084 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:30.326053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:30.390454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:30.390414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-755d6987f9-x4sck" event={"ID":"217d8208-48c8-4cd5-a19e-72fbdd288440","Type":"ContainerStarted","Data":"5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650"}
Sep 11 23:06:30.409978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:30.409908    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-755d6987f9-x4sck" podStartSLOduration=1.8067704949999999 podStartE2EDuration="4.409893896s" totalImagesPullingTime="2.603123401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:30.409465939 +0000 UTC m=+305.621437114" watchObservedRunningTime="2026-09-11 23:06:30.409893896 +0000 UTC m=+305.621865070"
Sep 11 23:06:31.294651 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:31.294618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:32.407794 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:32.407762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:06:32.536390 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:32.536354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:32.637707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:32.637670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:32.811604 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:32.811522    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:32.868000 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:32.867972    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:33.674043 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:33.674009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:33.771591 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:33.771558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:33.983498 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:33.983390    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.005858 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.005823    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"]
Sep 11 23:06:34.009101 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.009082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.011656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.011626    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 23:06:34.011656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.011649    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 23:06:34.011824 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.011700    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 23:06:34.012749 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.012732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 23:06:34.012828 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.012812    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-c5778\""
Sep 11 23:06:34.012945 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.012929    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 23:06:34.022483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.022461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"]
Sep 11 23:06:34.049037 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.049006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n94b8\" (UniqueName: \"kubernetes.io/projected/d1d6260f-708e-464d-b836-e92c47e9e8b5-kube-api-access-n94b8\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.049037 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.049041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.049300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.049142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.049300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.049222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6260f-708e-464d-b836-e92c47e9e8b5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.053584 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.053554    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z59sm"]
Sep 11 23:06:34.056629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.056611    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.059543 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.059523    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hcq8z\""
Sep 11 23:06:34.059799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.059778    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 23:06:34.059890 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.059781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 23:06:34.060208 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.060193    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 23:06:34.085655 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.085626    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.150585 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.150585 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-textfile\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.150847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qb8d\" (UniqueName: \"kubernetes.io/projected/c9353ba7-c74c-497b-b3dd-8092763474d9-kube-api-access-4qb8d\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.150847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-tls\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.150847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9353ba7-c74c-497b-b3dd-8092763474d9-metrics-client-ca\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.150847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6260f-708e-464d-b836-e92c47e9e8b5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.150847 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-root\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.151053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-wtmp\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.151053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n94b8\" (UniqueName: \"kubernetes.io/projected/d1d6260f-708e-464d-b836-e92c47e9e8b5-kube-api-access-n94b8\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.151053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.151053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.150991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.151053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.151021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-sys\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.151251 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.151053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.151251 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:06:34.151079    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 23:06:34.151251 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:06:34.151164    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-tls podName:d1d6260f-708e-464d-b836-e92c47e9e8b5 nodeName:}" failed. No retries permitted until 2026-09-11 23:06:34.651140707 +0000 UTC m=+309.863111858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-8vfbr" (UID: "d1d6260f-708e-464d-b836-e92c47e9e8b5") : secret "openshift-state-metrics-tls" not found
Sep 11 23:06:34.151482 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.151465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6260f-708e-464d-b836-e92c47e9e8b5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.153105 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.153084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.161714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.161689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n94b8\" (UniqueName: \"kubernetes.io/projected/d1d6260f-708e-464d-b836-e92c47e9e8b5-kube-api-access-n94b8\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.185395 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.185362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.252232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-textfile\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qb8d\" (UniqueName: \"kubernetes.io/projected/c9353ba7-c74c-497b-b3dd-8092763474d9-kube-api-access-4qb8d\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252503 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-tls\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252503 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9353ba7-c74c-497b-b3dd-8092763474d9-metrics-client-ca\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252503 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-root\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252503 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-wtmp\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252503 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:06:34.252400    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 23:06:34.252503 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:06:34.252491    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-tls podName:c9353ba7-c74c-497b-b3dd-8092763474d9 nodeName:}" failed. No retries permitted until 2026-09-11 23:06:34.75246731 +0000 UTC m=+309.964438481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-tls") pod "node-exporter-z59sm" (UID: "c9353ba7-c74c-497b-b3dd-8092763474d9") : secret "node-exporter-tls" not found
Sep 11 23:06:34.252503 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-root\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252819 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252819 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-sys\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252819 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-textfile\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252819 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-sys\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.252819 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-wtmp\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.253010 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.252836    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.253098 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.253018    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9353ba7-c74c-497b-b3dd-8092763474d9-metrics-client-ca\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.254802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.254783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.262677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.262655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qb8d\" (UniqueName: \"kubernetes.io/projected/c9353ba7-c74c-497b-b3dd-8092763474d9-kube-api-access-4qb8d\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.326046 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.326014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.426838 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.426809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.540785 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.540710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.640468 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.640434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.656707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.656680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.659133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.659105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1d6260f-708e-464d-b836-e92c47e9e8b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8vfbr\" (UID: \"d1d6260f-708e-464d-b836-e92c47e9e8b5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.757006 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.756985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-tls\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.759131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.759113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9353ba7-c74c-497b-b3dd-8092763474d9-node-exporter-tls\") pod \"node-exporter-z59sm\" (UID: \"c9353ba7-c74c-497b-b3dd-8092763474d9\") " pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.918521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.918491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"
Sep 11 23:06:34.935784 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.935758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:34.965861 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:34.965829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z59sm"
Sep 11 23:06:34.979931 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:06:34.979628    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9353ba7_c74c_497b_b3dd_8092763474d9.slice/crio-b2b3d597908e9d65d54d8d6f99c885854dac8bb0e06e1e7791061b1d2fe24774 WatchSource:0}: Error finding container b2b3d597908e9d65d54d8d6f99c885854dac8bb0e06e1e7791061b1d2fe24774: Status 404 returned error can't find the container with id b2b3d597908e9d65d54d8d6f99c885854dac8bb0e06e1e7791061b1d2fe24774
Sep 11 23:06:35.035883 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:35.035842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:35.191462 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:35.191381    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr"]
Sep 11 23:06:35.195299 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:06:35.195274    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d6260f_708e_464d_b836_e92c47e9e8b5.slice/crio-7ae8bc11717c191b6b2909530b257c9de03bc6c5495436c2d690bd0f9441fa22 WatchSource:0}: Error finding container 7ae8bc11717c191b6b2909530b257c9de03bc6c5495436c2d690bd0f9441fa22: Status 404 returned error can't find the container with id 7ae8bc11717c191b6b2909530b257c9de03bc6c5495436c2d690bd0f9441fa22
Sep 11 23:06:35.414350 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:35.414308    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z59sm" event={"ID":"c9353ba7-c74c-497b-b3dd-8092763474d9","Type":"ContainerStarted","Data":"b2b3d597908e9d65d54d8d6f99c885854dac8bb0e06e1e7791061b1d2fe24774"}
Sep 11 23:06:35.415219 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:35.415194    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr" event={"ID":"d1d6260f-708e-464d-b836-e92c47e9e8b5","Type":"ContainerStarted","Data":"edc51f123bdcdedbf664bfe407e8c86b34b77c5da2b2a7f5cb5cb7e26bd42f44"}
Sep 11 23:06:35.415304 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:35.415253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr" event={"ID":"d1d6260f-708e-464d-b836-e92c47e9e8b5","Type":"ContainerStarted","Data":"ddc1ef2cf67d878fcfca351ef346ee065bb5df73b1a5bd0baba60d4ad15130cc"}
Sep 11 23:06:35.415304 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:35.415281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr" event={"ID":"d1d6260f-708e-464d-b836-e92c47e9e8b5","Type":"ContainerStarted","Data":"7ae8bc11717c191b6b2909530b257c9de03bc6c5495436c2d690bd0f9441fa22"}
Sep 11 23:06:35.420126 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:35.420096    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:36.421391 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.421308    2580 generic.go:356] "Generic (PLEG): container finished" podID="c9353ba7-c74c-497b-b3dd-8092763474d9" containerID="095a1092388757f7d61d850cbd437c28dc55f7a30123da25b9d9afa85f7efdfb" exitCode=0
Sep 11 23:06:36.421391 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.421354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z59sm" event={"ID":"c9353ba7-c74c-497b-b3dd-8092763474d9","Type":"ContainerDied","Data":"095a1092388757f7d61d850cbd437c28dc55f7a30123da25b9d9afa85f7efdfb"}
Sep 11 23:06:36.600201 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.600155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:36.701282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.701192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:36.739058 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.739028    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:36.739201 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.739076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:36.743885 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.743864    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:36.913444 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:36.913414    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:37.016174 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.016094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:37.126692 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.126657    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:37.425852 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.425819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr" event={"ID":"d1d6260f-708e-464d-b836-e92c47e9e8b5","Type":"ContainerStarted","Data":"ab0b679e1020af00b5408078f993c7cd861cb9e38e150ee67a752215ed5ccd22"}
Sep 11 23:06:37.427816 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.427797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z59sm" event={"ID":"c9353ba7-c74c-497b-b3dd-8092763474d9","Type":"ContainerStarted","Data":"54eb6de3d88a8953acddcbaa4806d04edc2015db8adfbbe81ae7beaf8a2aa20b"}
Sep 11 23:06:37.427896 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.427843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z59sm" event={"ID":"c9353ba7-c74c-497b-b3dd-8092763474d9","Type":"ContainerStarted","Data":"2e3f2523c3dad2d72828324a87c986bdc542659fb4e9f9fa0d579e1eec7eecbc"}
Sep 11 23:06:37.431625 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.431608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:06:37.443259 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.443221    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8vfbr" podStartSLOduration=3.215298695 podStartE2EDuration="4.443205871s" totalImagesPullingTime="1.227907176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:37.442986219 +0000 UTC m=+312.654957392" watchObservedRunningTime="2026-09-11 23:06:37.443205871 +0000 UTC m=+312.655177046"
Sep 11 23:06:37.461744 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.461693    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z59sm" podStartSLOduration=2.5100664630000002 podStartE2EDuration="3.461677962s" totalImagesPullingTime="892.770263ms" totalInitContainerRuntime="58.841236ms" isStatefulPod=true podCreationTimestamp="2026-09-11 23:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:37.461022716 +0000 UTC m=+312.672993890" watchObservedRunningTime="2026-09-11 23:06:37.461677962 +0000 UTC m=+312.673649136"
Sep 11 23:06:37.577406 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:37.577351    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:38.634961 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:38.634907    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:38.720573 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:38.720540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:38.787403 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:38.787364    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:38.959273 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:38.959195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:40.216230 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:40.216195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:41.040132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.040094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:41.098865 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.098828    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:41.442000 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.441971    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-94678949c-k9bx7"]
Sep 11 23:06:41.447212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.447196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.457302 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.457280    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94678949c-k9bx7"]
Sep 11 23:06:41.511598 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.511562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b72d\" (UniqueName: \"kubernetes.io/projected/d1c0e1d5-22c9-4bc2-8646-54582340f16f-kube-api-access-8b72d\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.511598 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.511597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-serving-cert\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.511831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.511629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-oauth-serving-cert\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.511831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.511671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-service-ca\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.511831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.511729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-oauth-config\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.511831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.511766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-config\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.511831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.511790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-trusted-ca-bundle\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.612542 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.612512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b72d\" (UniqueName: \"kubernetes.io/projected/d1c0e1d5-22c9-4bc2-8646-54582340f16f-kube-api-access-8b72d\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.612542 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.612546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-serving-cert\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.612729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.612572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-oauth-serving-cert\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.612729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.612598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-service-ca\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.612729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.612617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-oauth-config\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.612729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.612637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-config\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.612729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.612662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-trusted-ca-bundle\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.613369 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.613341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-oauth-serving-cert\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.613478 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.613435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-service-ca\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.613532 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.613499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-trusted-ca-bundle\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.613571 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.613529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-config\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.614985 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.614966    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-serving-cert\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.615080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.615016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-oauth-config\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.620352 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.620330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b72d\" (UniqueName: \"kubernetes.io/projected/d1c0e1d5-22c9-4bc2-8646-54582340f16f-kube-api-access-8b72d\") pod \"console-94678949c-k9bx7\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") " pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:41.665897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.665873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:41.726650 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.726599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:41.755440 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:41.755422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:42.027946 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:42.027718    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94678949c-k9bx7"]
Sep 11 23:06:42.030684 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:06:42.030657    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c0e1d5_22c9_4bc2_8646_54582340f16f.slice/crio-1339592a4afbd13af10fbe8aa70a9addc1bad31b111ffb64415532bcb9dc0c43 WatchSource:0}: Error finding container 1339592a4afbd13af10fbe8aa70a9addc1bad31b111ffb64415532bcb9dc0c43: Status 404 returned error can't find the container with id 1339592a4afbd13af10fbe8aa70a9addc1bad31b111ffb64415532bcb9dc0c43
Sep 11 23:06:42.446516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:42.446479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94678949c-k9bx7" event={"ID":"d1c0e1d5-22c9-4bc2-8646-54582340f16f","Type":"ContainerStarted","Data":"4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd"}
Sep 11 23:06:42.446865 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:42.446553    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94678949c-k9bx7" event={"ID":"d1c0e1d5-22c9-4bc2-8646-54582340f16f","Type":"ContainerStarted","Data":"1339592a4afbd13af10fbe8aa70a9addc1bad31b111ffb64415532bcb9dc0c43"}
Sep 11 23:06:42.463328 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:42.463271    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-94678949c-k9bx7" podStartSLOduration=1.463254314 podStartE2EDuration="1.463254314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:42.462596011 +0000 UTC m=+317.674567184" watchObservedRunningTime="2026-09-11 23:06:42.463254314 +0000 UTC m=+317.675225489"
Sep 11 23:06:45.208855 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:45.208818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:45.309958 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:45.309879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:51.603617 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:51.603584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:51.664115 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:51.664073    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:51.755768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:51.755732    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:51.755960 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:51.755793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:51.760544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:51.760521    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:52.481120 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:52.481088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:06:52.530237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:52.530202    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-755d6987f9-x4sck"]
Sep 11 23:06:52.638871 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:52.638841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:52.694704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:52.694661    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:06:52.750606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:06:52.750528    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:06.609815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:06.609779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:06.669957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:06.669925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:09.312257 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:09.312210    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" podUID="7b256dd2-45b9-4708-97f8-f9ff82fea416" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 11 23:07:17.549996 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.549876    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-755d6987f9-x4sck" podUID="217d8208-48c8-4cd5-a19e-72fbdd288440" containerName="console" containerID="cri-o://5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650" gracePeriod=15
Sep 11 23:07:17.784664 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.784640    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:07:17.974671 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.974636    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tgpj\" (UniqueName: \"kubernetes.io/projected/217d8208-48c8-4cd5-a19e-72fbdd288440-kube-api-access-9tgpj\") pod \"217d8208-48c8-4cd5-a19e-72fbdd288440\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") "
Sep 11 23:07:17.974833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.974694    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-trusted-ca-bundle\") pod \"217d8208-48c8-4cd5-a19e-72fbdd288440\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") "
Sep 11 23:07:17.974833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.974715    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-oauth-config\") pod \"217d8208-48c8-4cd5-a19e-72fbdd288440\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") "
Sep 11 23:07:17.974833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.974730    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-oauth-serving-cert\") pod \"217d8208-48c8-4cd5-a19e-72fbdd288440\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") "
Sep 11 23:07:17.974833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.974754    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-service-ca\") pod \"217d8208-48c8-4cd5-a19e-72fbdd288440\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") "
Sep 11 23:07:17.975068 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.974874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-console-config\") pod \"217d8208-48c8-4cd5-a19e-72fbdd288440\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") "
Sep 11 23:07:17.975068 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.974957    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-serving-cert\") pod \"217d8208-48c8-4cd5-a19e-72fbdd288440\" (UID: \"217d8208-48c8-4cd5-a19e-72fbdd288440\") "
Sep 11 23:07:17.975495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.975173    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-service-ca" (OuterVolumeSpecName: "service-ca") pod "217d8208-48c8-4cd5-a19e-72fbdd288440" (UID: "217d8208-48c8-4cd5-a19e-72fbdd288440"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:17.975495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.975200    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "217d8208-48c8-4cd5-a19e-72fbdd288440" (UID: "217d8208-48c8-4cd5-a19e-72fbdd288440"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:17.975495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.975220    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "217d8208-48c8-4cd5-a19e-72fbdd288440" (UID: "217d8208-48c8-4cd5-a19e-72fbdd288440"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:17.975495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.975474    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-console-config" (OuterVolumeSpecName: "console-config") pod "217d8208-48c8-4cd5-a19e-72fbdd288440" (UID: "217d8208-48c8-4cd5-a19e-72fbdd288440"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:17.977101 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.977080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "217d8208-48c8-4cd5-a19e-72fbdd288440" (UID: "217d8208-48c8-4cd5-a19e-72fbdd288440"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:17.977180 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.977119    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/217d8208-48c8-4cd5-a19e-72fbdd288440-kube-api-access-9tgpj" (OuterVolumeSpecName: "kube-api-access-9tgpj") pod "217d8208-48c8-4cd5-a19e-72fbdd288440" (UID: "217d8208-48c8-4cd5-a19e-72fbdd288440"). InnerVolumeSpecName "kube-api-access-9tgpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:07:17.977212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:17.977183    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "217d8208-48c8-4cd5-a19e-72fbdd288440" (UID: "217d8208-48c8-4cd5-a19e-72fbdd288440"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:18.075884 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.075859    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-trusted-ca-bundle\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:07:18.075884 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.075880    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-oauth-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:07:18.075884 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.075890    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-oauth-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:07:18.076078 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.075898    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:07:18.076078 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.075907    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/217d8208-48c8-4cd5-a19e-72fbdd288440-console-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:07:18.076078 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.075929    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/217d8208-48c8-4cd5-a19e-72fbdd288440-console-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:07:18.076078 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.075937    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tgpj\" (UniqueName: \"kubernetes.io/projected/217d8208-48c8-4cd5-a19e-72fbdd288440-kube-api-access-9tgpj\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:07:18.568202 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.568167    2580 generic.go:356] "Generic (PLEG): container finished" podID="217d8208-48c8-4cd5-a19e-72fbdd288440" containerID="5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650" exitCode=0
Sep 11 23:07:18.568688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.568213    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-755d6987f9-x4sck"
Sep 11 23:07:18.568688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.568240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-755d6987f9-x4sck" event={"ID":"217d8208-48c8-4cd5-a19e-72fbdd288440","Type":"ContainerDied","Data":"5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650"}
Sep 11 23:07:18.568688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.568276    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-755d6987f9-x4sck" event={"ID":"217d8208-48c8-4cd5-a19e-72fbdd288440","Type":"ContainerDied","Data":"d29090ba86f5cb725fbda3a09da3a373393ca885d22faff1f4ea8583385fee86"}
Sep 11 23:07:18.568688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.568295    2580 scope.go:117] "RemoveContainer" containerID="5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650"
Sep 11 23:07:18.577145 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.577126    2580 scope.go:117] "RemoveContainer" containerID="5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650"
Sep 11 23:07:18.577434 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:07:18.577416    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650\": container with ID starting with 5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650 not found: ID does not exist" containerID="5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650"
Sep 11 23:07:18.577484 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.577444    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650"} err="failed to get container status \"5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650\": rpc error: code = NotFound desc = could not find container \"5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650\": container with ID starting with 5d8ddabdc261bc02c2dce02f194c80fd8bc7f36d40eee3fb5219417105298650 not found: ID does not exist"
Sep 11 23:07:18.588928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.588895    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-755d6987f9-x4sck"]
Sep 11 23:07:18.592435 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:18.592409    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-755d6987f9-x4sck"]
Sep 11 23:07:19.312622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:19.312586    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" podUID="7b256dd2-45b9-4708-97f8-f9ff82fea416" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 11 23:07:19.416169 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:19.413709    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="217d8208-48c8-4cd5-a19e-72fbdd288440" path="/var/lib/kubelet/pods/217d8208-48c8-4cd5-a19e-72fbdd288440/volumes"
Sep 11 23:07:23.033402 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:23.033356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:29.311719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:29.311681    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" podUID="7b256dd2-45b9-4708-97f8-f9ff82fea416" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 11 23:07:29.312105 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:29.311757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz"
Sep 11 23:07:29.312255 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:29.312238    2580 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"44915cc7aa19b7ba56a7fbb50224bbe40671dcf937be2941f4ec5fcbc0a763ea"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 11 23:07:29.312291 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:29.312277    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" podUID="7b256dd2-45b9-4708-97f8-f9ff82fea416" containerName="service-proxy" containerID="cri-o://44915cc7aa19b7ba56a7fbb50224bbe40671dcf937be2941f4ec5fcbc0a763ea" gracePeriod=30
Sep 11 23:07:29.612871 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:29.612838    2580 generic.go:356] "Generic (PLEG): container finished" podID="7b256dd2-45b9-4708-97f8-f9ff82fea416" containerID="44915cc7aa19b7ba56a7fbb50224bbe40671dcf937be2941f4ec5fcbc0a763ea" exitCode=2
Sep 11 23:07:29.613047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:29.612927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" event={"ID":"7b256dd2-45b9-4708-97f8-f9ff82fea416","Type":"ContainerDied","Data":"44915cc7aa19b7ba56a7fbb50224bbe40671dcf937be2941f4ec5fcbc0a763ea"}
Sep 11 23:07:29.613047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:29.612969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5647d8c496-l4lmz" event={"ID":"7b256dd2-45b9-4708-97f8-f9ff82fea416","Type":"ContainerStarted","Data":"ef700b691b35278af79995379e67dc35b2905d6434889938dfc99c7b41a869d9"}
Sep 11 23:07:31.869340 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:31.869305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:31.944673 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:31.944639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:34.082406 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:34.082373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:34.147769 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:34.147744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:39.218830 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:39.218796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:07:39.220986 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:39.220968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd3f622-28c8-42d7-bd7c-db92655889ba-metrics-certs\") pod \"network-metrics-daemon-9msgs\" (UID: \"8dd3f622-28c8-42d7-bd7c-db92655889ba\") " pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:07:39.312794 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:39.312768    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rtlm9\""
Sep 11 23:07:39.319623 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:39.319603    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9msgs"
Sep 11 23:07:39.586219 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:39.586189    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9msgs"]
Sep 11 23:07:39.589066 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:07:39.589033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd3f622_28c8_42d7_bd7c_db92655889ba.slice/crio-4a6eccaf76ff271c560440f8aaf7eebcca8032044c8b391c4d24b0a453885558 WatchSource:0}: Error finding container 4a6eccaf76ff271c560440f8aaf7eebcca8032044c8b391c4d24b0a453885558: Status 404 returned error can't find the container with id 4a6eccaf76ff271c560440f8aaf7eebcca8032044c8b391c4d24b0a453885558
Sep 11 23:07:39.644597 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:39.644571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9msgs" event={"ID":"8dd3f622-28c8-42d7-bd7c-db92655889ba","Type":"ContainerStarted","Data":"4a6eccaf76ff271c560440f8aaf7eebcca8032044c8b391c4d24b0a453885558"}
Sep 11 23:07:40.648944 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:40.648896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9msgs" event={"ID":"8dd3f622-28c8-42d7-bd7c-db92655889ba","Type":"ContainerStarted","Data":"20e3950b851a1c37351ee6cb6f45cd0aecf034f9e0de970e90f15ca87d5b57a9"}
Sep 11 23:07:41.653935 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:41.653878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9msgs" event={"ID":"8dd3f622-28c8-42d7-bd7c-db92655889ba","Type":"ContainerStarted","Data":"c315eeb0753bcd045057abae9cb362a16bf5dc542bb37c4d5e8b243cee4a0df2"}
Sep 11 23:07:41.671533 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:41.671487    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9msgs" podStartSLOduration=374.756050163 podStartE2EDuration="6m15.671471222s" totalImagesPullingTime="915.421059ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:07:41.671065464 +0000 UTC m=+376.883036638" watchObservedRunningTime="2026-09-11 23:07:41.671471222 +0000 UTC m=+376.883442395"
Sep 11 23:07:41.768730 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:41.768696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:41.840537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:41.840513    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:53.262265 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:53.262193    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:07:53.667143 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:07:53.667111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:03.015844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.015648    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67668f88bf-2sv2b"]
Sep 11 23:08:03.015844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.015870    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="217d8208-48c8-4cd5-a19e-72fbdd288440" containerName="console"
Sep 11 23:08:03.015844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.015882    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="217d8208-48c8-4cd5-a19e-72fbdd288440" containerName="console"
Sep 11 23:08:03.017701 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.015950    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="217d8208-48c8-4cd5-a19e-72fbdd288440" containerName="console"
Sep 11 23:08:03.020137 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.020113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.022598 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.022573    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67668f88bf-2sv2b"]
Sep 11 23:08:03.075075 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.075049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-trusted-ca-bundle\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.075214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.075080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-serving-cert\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.075214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.075102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-oauth-serving-cert\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.075214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.075167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-oauth-config\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.075214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.075207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-console-config\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.075396 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.075234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-service-ca\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.075396 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.075274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvhld\" (UniqueName: \"kubernetes.io/projected/2d3958af-a501-4096-a439-c0899da53daa-kube-api-access-jvhld\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.096491 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.096462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:03.156827 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.156799    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:03.176177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-serving-cert\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.176265 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-oauth-serving-cert\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.176265 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-oauth-config\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.176434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-console-config\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.176470 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-service-ca\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.176527 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvhld\" (UniqueName: \"kubernetes.io/projected/2d3958af-a501-4096-a439-c0899da53daa-kube-api-access-jvhld\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.176579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-trusted-ca-bundle\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.177015 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.176995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-oauth-serving-cert\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.177248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.177223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-service-ca\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.177327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.177234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-console-config\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.177327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.177296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-trusted-ca-bundle\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.178998 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.178977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-oauth-config\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.179190 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.179173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-serving-cert\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.185438 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.185419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvhld\" (UniqueName: \"kubernetes.io/projected/2d3958af-a501-4096-a439-c0899da53daa-kube-api-access-jvhld\") pod \"console-67668f88bf-2sv2b\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") " pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.217718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.217700    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:03.328762 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.328741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:03.599786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.599712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67668f88bf-2sv2b"]
Sep 11 23:08:03.603185 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:08:03.603156    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3958af_a501_4096_a439_c0899da53daa.slice/crio-808f223e4f41e761c96c11b9213978a7c8a8d72fb5db1d6fd0395feb2ea51267 WatchSource:0}: Error finding container 808f223e4f41e761c96c11b9213978a7c8a8d72fb5db1d6fd0395feb2ea51267: Status 404 returned error can't find the container with id 808f223e4f41e761c96c11b9213978a7c8a8d72fb5db1d6fd0395feb2ea51267
Sep 11 23:08:03.724794 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.724761    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67668f88bf-2sv2b" event={"ID":"2d3958af-a501-4096-a439-c0899da53daa","Type":"ContainerStarted","Data":"83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb"}
Sep 11 23:08:03.724959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.724809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67668f88bf-2sv2b" event={"ID":"2d3958af-a501-4096-a439-c0899da53daa","Type":"ContainerStarted","Data":"808f223e4f41e761c96c11b9213978a7c8a8d72fb5db1d6fd0395feb2ea51267"}
Sep 11 23:08:03.742371 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.742332    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67668f88bf-2sv2b" podStartSLOduration=1.742317873 podStartE2EDuration="1.742317873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:08:03.741737219 +0000 UTC m=+398.953708392" watchObservedRunningTime="2026-09-11 23:08:03.742317873 +0000 UTC m=+398.954289048"
Sep 11 23:08:03.805056 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:03.805021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:13.329850 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:13.329813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:13.330414 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:13.329873    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:13.334767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:13.334741    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:13.761938 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:13.761848    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:08:13.813457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:13.813425    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94678949c-k9bx7"]
Sep 11 23:08:13.930886 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:13.930851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:14.031352 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:14.031253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:14.083964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:14.083937    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:38.835109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:38.835054    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-94678949c-k9bx7" podUID="d1c0e1d5-22c9-4bc2-8646-54582340f16f" containerName="console" containerID="cri-o://4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd" gracePeriod=15
Sep 11 23:08:39.060696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.060671    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:08:39.122473 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122407    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-config\") pod \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") "
Sep 11 23:08:39.122473 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-oauth-config\") pod \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") "
Sep 11 23:08:39.122661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122480    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b72d\" (UniqueName: \"kubernetes.io/projected/d1c0e1d5-22c9-4bc2-8646-54582340f16f-kube-api-access-8b72d\") pod \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") "
Sep 11 23:08:39.122661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122496    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-trusted-ca-bundle\") pod \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") "
Sep 11 23:08:39.122661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122519    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-oauth-serving-cert\") pod \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") "
Sep 11 23:08:39.122661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122554    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-serving-cert\") pod \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") "
Sep 11 23:08:39.122661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122573    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-service-ca\") pod \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\" (UID: \"d1c0e1d5-22c9-4bc2-8646-54582340f16f\") "
Sep 11 23:08:39.122982 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.122954    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-config" (OuterVolumeSpecName: "console-config") pod "d1c0e1d5-22c9-4bc2-8646-54582340f16f" (UID: "d1c0e1d5-22c9-4bc2-8646-54582340f16f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:08:39.123044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.123016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d1c0e1d5-22c9-4bc2-8646-54582340f16f" (UID: "d1c0e1d5-22c9-4bc2-8646-54582340f16f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:08:39.123044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.123010    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d1c0e1d5-22c9-4bc2-8646-54582340f16f" (UID: "d1c0e1d5-22c9-4bc2-8646-54582340f16f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:08:39.123359 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.123118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-service-ca" (OuterVolumeSpecName: "service-ca") pod "d1c0e1d5-22c9-4bc2-8646-54582340f16f" (UID: "d1c0e1d5-22c9-4bc2-8646-54582340f16f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:08:39.124651 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.124613    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d1c0e1d5-22c9-4bc2-8646-54582340f16f" (UID: "d1c0e1d5-22c9-4bc2-8646-54582340f16f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:08:39.124757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.124668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d1c0e1d5-22c9-4bc2-8646-54582340f16f" (UID: "d1c0e1d5-22c9-4bc2-8646-54582340f16f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:08:39.124757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.124677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c0e1d5-22c9-4bc2-8646-54582340f16f-kube-api-access-8b72d" (OuterVolumeSpecName: "kube-api-access-8b72d") pod "d1c0e1d5-22c9-4bc2-8646-54582340f16f" (UID: "d1c0e1d5-22c9-4bc2-8646-54582340f16f"). InnerVolumeSpecName "kube-api-access-8b72d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:08:39.223499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.223472    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b72d\" (UniqueName: \"kubernetes.io/projected/d1c0e1d5-22c9-4bc2-8646-54582340f16f-kube-api-access-8b72d\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:08:39.223499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.223495    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-trusted-ca-bundle\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:08:39.223674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.223505    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-oauth-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:08:39.223674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.223515    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:08:39.223674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.223524    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:08:39.223674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.223533    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:08:39.223674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.223540    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d1c0e1d5-22c9-4bc2-8646-54582340f16f-console-oauth-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:08:39.845737 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.845698    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1c0e1d5-22c9-4bc2-8646-54582340f16f" containerID="4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd" exitCode=0
Sep 11 23:08:39.846119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.845762    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94678949c-k9bx7"
Sep 11 23:08:39.846119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.845790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94678949c-k9bx7" event={"ID":"d1c0e1d5-22c9-4bc2-8646-54582340f16f","Type":"ContainerDied","Data":"4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd"}
Sep 11 23:08:39.846119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.845837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94678949c-k9bx7" event={"ID":"d1c0e1d5-22c9-4bc2-8646-54582340f16f","Type":"ContainerDied","Data":"1339592a4afbd13af10fbe8aa70a9addc1bad31b111ffb64415532bcb9dc0c43"}
Sep 11 23:08:39.846119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.845858    2580 scope.go:117] "RemoveContainer" containerID="4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd"
Sep 11 23:08:39.853523 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.853507    2580 scope.go:117] "RemoveContainer" containerID="4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd"
Sep 11 23:08:39.853788 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:08:39.853770    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd\": container with ID starting with 4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd not found: ID does not exist" containerID="4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd"
Sep 11 23:08:39.853842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.853797    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd"} err="failed to get container status \"4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd\": rpc error: code = NotFound desc = could not find container \"4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd\": container with ID starting with 4f3a7a7729381b9b8b4131d2b6fab7283ea0cc2d6e05146c0a946a3cd281a5bd not found: ID does not exist"
Sep 11 23:08:39.862863 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.862839    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94678949c-k9bx7"]
Sep 11 23:08:39.866478 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:39.866458    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-94678949c-k9bx7"]
Sep 11 23:08:41.411861 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:41.411827    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1c0e1d5-22c9-4bc2-8646-54582340f16f" path="/var/lib/kubelet/pods/d1c0e1d5-22c9-4bc2-8646-54582340f16f/volumes"
Sep 11 23:08:42.626465 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:42.626428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:08:42.679114 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:08:42.679088    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:08.536245 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.536211    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dbcccc57d-flhsf"]
Sep 11 23:09:08.536712 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.536455    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1c0e1d5-22c9-4bc2-8646-54582340f16f" containerName="console"
Sep 11 23:09:08.536712 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.536468    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1c0e1d5-22c9-4bc2-8646-54582340f16f" containerName="console"
Sep 11 23:09:08.536712 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.536516    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1c0e1d5-22c9-4bc2-8646-54582340f16f" containerName="console"
Sep 11 23:09:08.539200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.539175    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.549686 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.549662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbcccc57d-flhsf"]
Sep 11 23:09:08.553002 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.552974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:08.620612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.620590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhsq\" (UniqueName: \"kubernetes.io/projected/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-kube-api-access-5qhsq\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.620724 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.620630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-oauth-serving-cert\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.620724 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.620648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-serving-cert\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.620797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.620725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-service-ca\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.620797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.620749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-trusted-ca-bundle\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.620797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.620776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-oauth-config\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.620888 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.620800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-config\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.653592 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.653568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:08.721845 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.721805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-oauth-serving-cert\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.721845 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.721849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-serving-cert\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722019 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.721883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-service-ca\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722019 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.721899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-trusted-ca-bundle\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722019 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.721970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-oauth-config\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722019 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.722000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-config\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.722023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhsq\" (UniqueName: \"kubernetes.io/projected/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-kube-api-access-5qhsq\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.722672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-service-ca\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722796 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.722672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-oauth-serving-cert\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.722829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-trusted-ca-bundle\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.722948 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.722869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-config\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.724370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.724342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-oauth-config\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.724468 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.724449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-serving-cert\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.730825 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.730806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhsq\" (UniqueName: \"kubernetes.io/projected/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-kube-api-access-5qhsq\") pod \"console-dbcccc57d-flhsf\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") " pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.761894 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.761874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:08.848280 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.848251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:08.855067 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:08.855021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:09.118493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:09.118418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbcccc57d-flhsf"]
Sep 11 23:09:09.121604 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:09:09.121578    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569cddb9_a33d_49cb_a14d_4c68d6c6cbe2.slice/crio-acb21622559d03b016586e79eeb86e8bc60d029186b29d706e270a882ad6c6ec WatchSource:0}: Error finding container acb21622559d03b016586e79eeb86e8bc60d029186b29d706e270a882ad6c6ec: Status 404 returned error can't find the container with id acb21622559d03b016586e79eeb86e8bc60d029186b29d706e270a882ad6c6ec
Sep 11 23:09:09.550678 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:09.550583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:09.652118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:09.652079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:09.947171 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:09.947135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcccc57d-flhsf" event={"ID":"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2","Type":"ContainerStarted","Data":"c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f"}
Sep 11 23:09:09.947315 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:09.947191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcccc57d-flhsf" event={"ID":"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2","Type":"ContainerStarted","Data":"acb21622559d03b016586e79eeb86e8bc60d029186b29d706e270a882ad6c6ec"}
Sep 11 23:09:09.985684 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:09.985642    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dbcccc57d-flhsf" podStartSLOduration=1.9856263969999999 podStartE2EDuration="1.985626397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:09:09.984051939 +0000 UTC m=+465.196023113" watchObservedRunningTime="2026-09-11 23:09:09.985626397 +0000 UTC m=+465.197597570"
Sep 11 23:09:18.849398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:18.849351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:18.849829 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:18.849448    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:18.853874 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:18.853851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:18.981856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:18.981827    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:09:19.035950 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:19.035902    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67668f88bf-2sv2b"]
Sep 11 23:09:19.194438 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:19.194356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:19.279855 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:19.279823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:20.358417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:20.358382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:20.460642 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:20.460606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:20.870061 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:20.870026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:20.936119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:20.936086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:36.626305 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:36.626267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:36.727629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:36.727593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:37.055103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:37.055018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:09:44.054579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.054542    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67668f88bf-2sv2b" podUID="2d3958af-a501-4096-a439-c0899da53daa" containerName="console" containerID="cri-o://83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb" gracePeriod=15
Sep 11 23:09:44.288554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.288530    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:09:44.372857 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.372828    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-trusted-ca-bundle\") pod \"2d3958af-a501-4096-a439-c0899da53daa\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") "
Sep 11 23:09:44.373021 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.372863    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-serving-cert\") pod \"2d3958af-a501-4096-a439-c0899da53daa\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") "
Sep 11 23:09:44.373021 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.372881    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvhld\" (UniqueName: \"kubernetes.io/projected/2d3958af-a501-4096-a439-c0899da53daa-kube-api-access-jvhld\") pod \"2d3958af-a501-4096-a439-c0899da53daa\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") "
Sep 11 23:09:44.373021 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.372899    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-oauth-serving-cert\") pod \"2d3958af-a501-4096-a439-c0899da53daa\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") "
Sep 11 23:09:44.373156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-console-config\") pod \"2d3958af-a501-4096-a439-c0899da53daa\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") "
Sep 11 23:09:44.373211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373157    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-oauth-config\") pod \"2d3958af-a501-4096-a439-c0899da53daa\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") "
Sep 11 23:09:44.373211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-service-ca\") pod \"2d3958af-a501-4096-a439-c0899da53daa\" (UID: \"2d3958af-a501-4096-a439-c0899da53daa\") "
Sep 11 23:09:44.373309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373272    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2d3958af-a501-4096-a439-c0899da53daa" (UID: "2d3958af-a501-4096-a439-c0899da53daa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:09:44.373360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373291    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2d3958af-a501-4096-a439-c0899da53daa" (UID: "2d3958af-a501-4096-a439-c0899da53daa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:09:44.373420 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-console-config" (OuterVolumeSpecName: "console-config") pod "2d3958af-a501-4096-a439-c0899da53daa" (UID: "2d3958af-a501-4096-a439-c0899da53daa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:09:44.373581 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373559    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-trusted-ca-bundle\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:09:44.373581 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373581    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-oauth-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:09:44.373714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.373676    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-service-ca" (OuterVolumeSpecName: "service-ca") pod "2d3958af-a501-4096-a439-c0899da53daa" (UID: "2d3958af-a501-4096-a439-c0899da53daa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:09:44.375103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.375073    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3958af-a501-4096-a439-c0899da53daa-kube-api-access-jvhld" (OuterVolumeSpecName: "kube-api-access-jvhld") pod "2d3958af-a501-4096-a439-c0899da53daa" (UID: "2d3958af-a501-4096-a439-c0899da53daa"). InnerVolumeSpecName "kube-api-access-jvhld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:09:44.375196 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.375171    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2d3958af-a501-4096-a439-c0899da53daa" (UID: "2d3958af-a501-4096-a439-c0899da53daa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:09:44.375241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.375187    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2d3958af-a501-4096-a439-c0899da53daa" (UID: "2d3958af-a501-4096-a439-c0899da53daa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:09:44.474098 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.474061    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:09:44.474098 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.474098    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvhld\" (UniqueName: \"kubernetes.io/projected/2d3958af-a501-4096-a439-c0899da53daa-kube-api-access-jvhld\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:09:44.474238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.474115    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-console-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:09:44.474238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.474129    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d3958af-a501-4096-a439-c0899da53daa-console-oauth-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:09:44.474238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:44.474141    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3958af-a501-4096-a439-c0899da53daa-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:09:45.070688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.070650    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d3958af-a501-4096-a439-c0899da53daa" containerID="83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb" exitCode=0
Sep 11 23:09:45.071095 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.070694    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67668f88bf-2sv2b"
Sep 11 23:09:45.071095 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.070730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67668f88bf-2sv2b" event={"ID":"2d3958af-a501-4096-a439-c0899da53daa","Type":"ContainerDied","Data":"83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb"}
Sep 11 23:09:45.071095 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.070779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67668f88bf-2sv2b" event={"ID":"2d3958af-a501-4096-a439-c0899da53daa","Type":"ContainerDied","Data":"808f223e4f41e761c96c11b9213978a7c8a8d72fb5db1d6fd0395feb2ea51267"}
Sep 11 23:09:45.071095 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.070798    2580 scope.go:117] "RemoveContainer" containerID="83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb"
Sep 11 23:09:45.078788 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.078640    2580 scope.go:117] "RemoveContainer" containerID="83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb"
Sep 11 23:09:45.078996 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:09:45.078973    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb\": container with ID starting with 83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb not found: ID does not exist" containerID="83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb"
Sep 11 23:09:45.079047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.079005    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb"} err="failed to get container status \"83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb\": rpc error: code = NotFound desc = could not find container \"83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb\": container with ID starting with 83f453e0f5c8481b11286c319c670a4512e177579f4e6b0b582f116d6558fcbb not found: ID does not exist"
Sep 11 23:09:45.092143 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.092122    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67668f88bf-2sv2b"]
Sep 11 23:09:45.097644 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.097623    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67668f88bf-2sv2b"]
Sep 11 23:09:45.412439 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:09:45.412366    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3958af-a501-4096-a439-c0899da53daa" path="/var/lib/kubelet/pods/2d3958af-a501-4096-a439-c0899da53daa/volumes"
Sep 11 23:10:21.820452 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:21.820373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:21.905321 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:21.905287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:43.901454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:43.901401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:45.298414 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:45.298365    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:47.524793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:47.524756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:47.625160 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:47.625116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:48.991136 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:48.991099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:49.108239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:49.108193    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:49.373118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:49.373082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:49.502481 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:49.502454    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:51.697145 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:51.697116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:51.907298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:51.907257    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:54.308127 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:54.308081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:54.370871 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:54.370842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:57.264360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:57.264327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:57.324672 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:57.324635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:59.146650 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.146611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:10:59.849821 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.849785    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rrl8x"]
Sep 11 23:10:59.850031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.850019    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d3958af-a501-4096-a439-c0899da53daa" containerName="console"
Sep 11 23:10:59.850081 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.850033    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3958af-a501-4096-a439-c0899da53daa" containerName="console"
Sep 11 23:10:59.850119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.850085    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d3958af-a501-4096-a439-c0899da53daa" containerName="console"
Sep 11 23:10:59.852706 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.852687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:10:59.855503 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.855475    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 23:10:59.855609 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.855517    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 23:10:59.856737 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.856721    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-fhzlt\""
Sep 11 23:10:59.864468 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.864435    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rrl8x"]
Sep 11 23:10:59.984513 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.984432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f139dbc-7ebc-49a7-942e-4de52202ed2b-bound-sa-token\") pod \"cert-manager-55b4d954d5-rrl8x\" (UID: \"7f139dbc-7ebc-49a7-942e-4de52202ed2b\") " pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:10:59.984642 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:10:59.984550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55z47\" (UniqueName: \"kubernetes.io/projected/7f139dbc-7ebc-49a7-942e-4de52202ed2b-kube-api-access-55z47\") pod \"cert-manager-55b4d954d5-rrl8x\" (UID: \"7f139dbc-7ebc-49a7-942e-4de52202ed2b\") " pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:11:00.085781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:00.085731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f139dbc-7ebc-49a7-942e-4de52202ed2b-bound-sa-token\") pod \"cert-manager-55b4d954d5-rrl8x\" (UID: \"7f139dbc-7ebc-49a7-942e-4de52202ed2b\") " pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:11:00.085781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:00.085783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55z47\" (UniqueName: \"kubernetes.io/projected/7f139dbc-7ebc-49a7-942e-4de52202ed2b-kube-api-access-55z47\") pod \"cert-manager-55b4d954d5-rrl8x\" (UID: \"7f139dbc-7ebc-49a7-942e-4de52202ed2b\") " pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:11:00.094802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:00.094776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f139dbc-7ebc-49a7-942e-4de52202ed2b-bound-sa-token\") pod \"cert-manager-55b4d954d5-rrl8x\" (UID: \"7f139dbc-7ebc-49a7-942e-4de52202ed2b\") " pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:11:00.094952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:00.094935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55z47\" (UniqueName: \"kubernetes.io/projected/7f139dbc-7ebc-49a7-942e-4de52202ed2b-kube-api-access-55z47\") pod \"cert-manager-55b4d954d5-rrl8x\" (UID: \"7f139dbc-7ebc-49a7-942e-4de52202ed2b\") " pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:11:00.161801 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:00.161718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rrl8x"
Sep 11 23:11:00.427945 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:00.427852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rrl8x"]
Sep 11 23:11:00.430756 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:11:00.430728    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f139dbc_7ebc_49a7_942e_4de52202ed2b.slice/crio-a863f673847a6398a98074b4e8daf9537e5201e58cb0d8edceea5e0bb9519724 WatchSource:0}: Error finding container a863f673847a6398a98074b4e8daf9537e5201e58cb0d8edceea5e0bb9519724: Status 404 returned error can't find the container with id a863f673847a6398a98074b4e8daf9537e5201e58cb0d8edceea5e0bb9519724
Sep 11 23:11:01.325968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:01.325927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rrl8x" event={"ID":"7f139dbc-7ebc-49a7-942e-4de52202ed2b","Type":"ContainerStarted","Data":"a863f673847a6398a98074b4e8daf9537e5201e58cb0d8edceea5e0bb9519724"}
Sep 11 23:11:02.717144 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:02.717104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:02.821745 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:02.821704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:04.339009 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:04.338973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rrl8x" event={"ID":"7f139dbc-7ebc-49a7-942e-4de52202ed2b","Type":"ContainerStarted","Data":"86a4e501a8fcf7c20185b734af66b76faeb29b214e0a1245e5face95b78c3ed3"}
Sep 11 23:11:04.358430 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:04.358380    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rrl8x" podStartSLOduration=2.359973724 podStartE2EDuration="5.358365269s" totalImagesPullingTime="2.998391545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:11:04.356361248 +0000 UTC m=+579.568332421" watchObservedRunningTime="2026-09-11 23:11:04.358365269 +0000 UTC m=+579.570336442"
Sep 11 23:11:08.522220 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:08.522177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:08.622358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:08.622318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:10.195927 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:10.195877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:10.389025 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:10.388978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:19.698144 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.698109    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-588798ddb-bvhk6"]
Sep 11 23:11:19.701175 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.701155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.710925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.710890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588798ddb-bvhk6"]
Sep 11 23:11:19.789610 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.789571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:19.820043 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.820021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-trusted-ca-bundle\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.820140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.820053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-oauth-serving-cert\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.820140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.820072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f84190e-7954-470a-a346-2dd304ecebc6-console-oauth-config\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.820140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.820097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dzz8\" (UniqueName: \"kubernetes.io/projected/3f84190e-7954-470a-a346-2dd304ecebc6-kube-api-access-7dzz8\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.820232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.820159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-console-config\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.820232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.820192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f84190e-7954-470a-a346-2dd304ecebc6-console-serving-cert\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.820232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.820211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-service-ca\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.855417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.855392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:19.911042 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.911012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:19.921485 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.921464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dzz8\" (UniqueName: \"kubernetes.io/projected/3f84190e-7954-470a-a346-2dd304ecebc6-kube-api-access-7dzz8\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.921573 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.921493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-console-config\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.921573 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.921515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f84190e-7954-470a-a346-2dd304ecebc6-console-serving-cert\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.921573 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.921530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-service-ca\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.921709 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.921663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-trusted-ca-bundle\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.921763 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.921716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-oauth-serving-cert\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.921763 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.921744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f84190e-7954-470a-a346-2dd304ecebc6-console-oauth-config\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.922337 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.922306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-console-config\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.922447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.922334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-service-ca\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.922511 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.922468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-oauth-serving-cert\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.922574 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.922516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f84190e-7954-470a-a346-2dd304ecebc6-trusted-ca-bundle\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.924055 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.924033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f84190e-7954-470a-a346-2dd304ecebc6-console-serving-cert\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.924156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.924119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f84190e-7954-470a-a346-2dd304ecebc6-console-oauth-config\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:19.930429 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:19.930408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dzz8\" (UniqueName: \"kubernetes.io/projected/3f84190e-7954-470a-a346-2dd304ecebc6-kube-api-access-7dzz8\") pod \"console-588798ddb-bvhk6\" (UID: \"3f84190e-7954-470a-a346-2dd304ecebc6\") " pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:20.009418 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:20.009361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:20.278981 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:20.278958    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588798ddb-bvhk6"]
Sep 11 23:11:20.281275 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:11:20.281249    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f84190e_7954_470a_a346_2dd304ecebc6.slice/crio-4f25db6f431349e5ee0b39846b813ad8bc3fc87b83cf782e6a48fe4c54e1ae7c WatchSource:0}: Error finding container 4f25db6f431349e5ee0b39846b813ad8bc3fc87b83cf782e6a48fe4c54e1ae7c: Status 404 returned error can't find the container with id 4f25db6f431349e5ee0b39846b813ad8bc3fc87b83cf782e6a48fe4c54e1ae7c
Sep 11 23:11:20.401526 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:20.401489    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588798ddb-bvhk6" event={"ID":"3f84190e-7954-470a-a346-2dd304ecebc6","Type":"ContainerStarted","Data":"ae58520515f9d175dbdb445539d9229123a70e11ef261604fcb08327a5292a0f"}
Sep 11 23:11:20.401669 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:20.401578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588798ddb-bvhk6" event={"ID":"3f84190e-7954-470a-a346-2dd304ecebc6","Type":"ContainerStarted","Data":"4f25db6f431349e5ee0b39846b813ad8bc3fc87b83cf782e6a48fe4c54e1ae7c"}
Sep 11 23:11:20.419848 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:20.419801    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-588798ddb-bvhk6" podStartSLOduration=1.419787149 podStartE2EDuration="1.419787149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:11:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:11:20.418779357 +0000 UTC m=+595.630750531" watchObservedRunningTime="2026-09-11 23:11:20.419787149 +0000 UTC m=+595.631758372"
Sep 11 23:11:20.493099 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:20.493068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:20.549832 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:20.549771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:26.316425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:26.316363    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:27.329709 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.329651    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"]
Sep 11 23:11:27.332825 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.332805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.335596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.335575    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 23:11:27.335716 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.335697    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 23:11:27.335865 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.335847    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 23:11:27.335945 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.335868    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 23:11:27.335945 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.335851    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2lx4s\""
Sep 11 23:11:27.367244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.367223    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"]
Sep 11 23:11:27.477156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.477127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e0139f08-8083-4e56-bd5c-d0a328a9a18f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.477282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.477176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0139f08-8083-4e56-bd5c-d0a328a9a18f-tmp\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.477282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.477199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0139f08-8083-4e56-bd5c-d0a328a9a18f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.477282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.477234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8p7\" (UniqueName: \"kubernetes.io/projected/e0139f08-8083-4e56-bd5c-d0a328a9a18f-kube-api-access-ng8p7\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.477282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.477255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0139f08-8083-4e56-bd5c-d0a328a9a18f-manifests\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.578420 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.578391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8p7\" (UniqueName: \"kubernetes.io/projected/e0139f08-8083-4e56-bd5c-d0a328a9a18f-kube-api-access-ng8p7\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.578420 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.578427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0139f08-8083-4e56-bd5c-d0a328a9a18f-manifests\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.578628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.578457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e0139f08-8083-4e56-bd5c-d0a328a9a18f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.578628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.578512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0139f08-8083-4e56-bd5c-d0a328a9a18f-tmp\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.578628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.578550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0139f08-8083-4e56-bd5c-d0a328a9a18f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.578880 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.578857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e0139f08-8083-4e56-bd5c-d0a328a9a18f-manifests\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.580806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.580760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0139f08-8083-4e56-bd5c-d0a328a9a18f-tmp\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.581038 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.581022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e0139f08-8083-4e56-bd5c-d0a328a9a18f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.581148 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.581126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e0139f08-8083-4e56-bd5c-d0a328a9a18f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.589925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.589892    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8p7\" (UniqueName: \"kubernetes.io/projected/e0139f08-8083-4e56-bd5c-d0a328a9a18f-kube-api-access-ng8p7\") pod \"opendatahub-operator-controller-manager-6857545767-sftvs\" (UID: \"e0139f08-8083-4e56-bd5c-d0a328a9a18f\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.642002 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.641983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:27.917226 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.917130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:27.918014 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.917992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"]
Sep 11 23:11:27.920379 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:11:27.920352    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0139f08_8083_4e56_bd5c_d0a328a9a18f.slice/crio-20b6dfd1e7d65960896cee6d02b4152875c6c0a75d218fb55cb244db75d22dfe WatchSource:0}: Error finding container 20b6dfd1e7d65960896cee6d02b4152875c6c0a75d218fb55cb244db75d22dfe: Status 404 returned error can't find the container with id 20b6dfd1e7d65960896cee6d02b4152875c6c0a75d218fb55cb244db75d22dfe
Sep 11 23:11:27.922157 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:27.922138    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:11:28.010628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:28.010599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:28.111181 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:28.111140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:28.429199 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:28.429162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs" event={"ID":"e0139f08-8083-4e56-bd5c-d0a328a9a18f","Type":"ContainerStarted","Data":"20b6dfd1e7d65960896cee6d02b4152875c6c0a75d218fb55cb244db75d22dfe"}
Sep 11 23:11:30.010507 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.010468    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:30.010957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.010539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:30.015836 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.015812    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:30.441305 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.441276    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-588798ddb-bvhk6"
Sep 11 23:11:30.490022 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.489977    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dbcccc57d-flhsf"]
Sep 11 23:11:30.640371 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.640332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:30.743961 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.743858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:30.848389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:30.848355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:31.442483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:31.442452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs" event={"ID":"e0139f08-8083-4e56-bd5c-d0a328a9a18f","Type":"ContainerStarted","Data":"049f99749a4dc18b7c50b0ade7f27b5169454aaa893714ab880861c32cb71a1e"}
Sep 11 23:11:32.448150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:32.448120    2580 generic.go:356] "Generic (PLEG): container finished" podID="e0139f08-8083-4e56-bd5c-d0a328a9a18f" containerID="049f99749a4dc18b7c50b0ade7f27b5169454aaa893714ab880861c32cb71a1e" exitCode=0
Sep 11 23:11:32.448525 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:32.448199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs" event={"ID":"e0139f08-8083-4e56-bd5c-d0a328a9a18f","Type":"ContainerDied","Data":"049f99749a4dc18b7c50b0ade7f27b5169454aaa893714ab880861c32cb71a1e"}
Sep 11 23:11:33.453742 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:33.453709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs" event={"ID":"e0139f08-8083-4e56-bd5c-d0a328a9a18f","Type":"ContainerStarted","Data":"070a31b71bd3a346532296a9d5264ed2ec6926fafca1693c775cb1baf7fc7045"}
Sep 11 23:11:33.454132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:33.453820    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:33.477567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:33.477518    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs" podStartSLOduration=2.05105057 podStartE2EDuration="6.477499658s" totalImagesPullingTime="3.382563165s" totalInitContainerRuntime="1.043885923s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:11:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:11:33.475742407 +0000 UTC m=+608.687713582" watchObservedRunningTime="2026-09-11 23:11:33.477499658 +0000 UTC m=+608.689470832"
Sep 11 23:11:34.654681 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:34.654632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:34.755500 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:34.755467    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:39.534192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:39.534163    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:39.639280 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:39.639249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:43.410281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:43.410244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:43.513276 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:43.513241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:44.459436 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:44.459405    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-sftvs"
Sep 11 23:11:44.522258 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:44.522218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:44.635605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:44.635575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:44.964342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:44.964310    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:45.059747 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:45.059718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:46.659364 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:46.659327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:46.777402 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:46.777367    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:47.768733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.768649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:47.803236 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.803187    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"]
Sep 11 23:11:47.812242 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.812222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:47.815255 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.815219    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 23:11:47.815387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.815240    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 23:11:47.815387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.815275    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 23:11:47.815510 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.815452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"]
Sep 11 23:11:47.815510 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.815463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5jbkg\""
Sep 11 23:11:47.815588 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.815540    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 23:11:47.922652 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.922628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl8xq\" (UniqueName: \"kubernetes.io/projected/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-kube-api-access-tl8xq\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:47.922790 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.922661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tmp\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:47.922790 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:47.922703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tls-certs\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.023431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.023378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tls-certs\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.023431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.023423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl8xq\" (UniqueName: \"kubernetes.io/projected/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-kube-api-access-tl8xq\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.023545 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.023449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tmp\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.023545 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:11:48.023509    2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 11 23:11:48.023628 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:11:48.023579    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tls-certs podName:2d6d34ee-6d2c-4771-9874-7784fd39dd3f nodeName:}" failed. No retries permitted until 2026-09-11 23:11:48.523562556 +0000 UTC m=+623.735533708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tls-certs") pod "kube-auth-proxy-7c75bd8f97-b9d4w" (UID: "2d6d34ee-6d2c-4771-9874-7784fd39dd3f") : secret "kube-auth-proxy-tls" not found
Sep 11 23:11:48.025543 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.025525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tmp\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.032198 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.032176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl8xq\" (UniqueName: \"kubernetes.io/projected/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-kube-api-access-tl8xq\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.471940 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.469090    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:48.527574 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.527539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tls-certs\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.529869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.529845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d34ee-6d2c-4771-9874-7784fd39dd3f-tls-certs\") pod \"kube-auth-proxy-7c75bd8f97-b9d4w\" (UID: \"2d6d34ee-6d2c-4771-9874-7784fd39dd3f\") " pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.722837 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.722767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"
Sep 11 23:11:48.993775 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:48.993709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w"]
Sep 11 23:11:48.996417 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:11:48.996388    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6d34ee_6d2c_4771_9874_7784fd39dd3f.slice/crio-fc295e6c31936dcab1241e5bb908131bc466ed7b7fe1573f025cf215351e3592 WatchSource:0}: Error finding container fc295e6c31936dcab1241e5bb908131bc466ed7b7fe1573f025cf215351e3592: Status 404 returned error can't find the container with id fc295e6c31936dcab1241e5bb908131bc466ed7b7fe1573f025cf215351e3592
Sep 11 23:11:49.172239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.172204    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"]
Sep 11 23:11:49.173786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.173752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:49.176650 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.176635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.179558 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.179536    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h4lwq\""
Sep 11 23:11:49.179656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.179542    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 23:11:49.179656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.179627    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 23:11:49.179797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.179702    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 23:11:49.179951 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.179935    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 23:11:49.190891 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.190870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"]
Sep 11 23:11:49.231849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.231826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.231971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.231867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.231971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.231960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.232052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.231991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzlzg\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-kube-api-access-vzlzg\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.232052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.232033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.232112 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.232050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaa15d84-573a-4116-b92f-814de5f0b758-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.232112 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.232071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.263347 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.263284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:49.332609 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.332585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.332714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.332626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.332775 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.332740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzlzg\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-kube-api-access-vzlzg\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.332834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.332775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.332834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.332792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaa15d84-573a-4116-b92f-814de5f0b758-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.332973 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.332934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.333032 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.332985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.333271 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.333253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.335260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.335236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.335365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.335266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.335515 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.335493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaa15d84-573a-4116-b92f-814de5f0b758-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.335607 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.335587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.340736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.340710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.341132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.341115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzlzg\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-kube-api-access-vzlzg\") pod \"istiod-openshift-gateway-7b7995d5cd-r4fsw\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.486282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.486253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:49.510331 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.510282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w" event={"ID":"2d6d34ee-6d2c-4771-9874-7784fd39dd3f","Type":"ContainerStarted","Data":"fc295e6c31936dcab1241e5bb908131bc466ed7b7fe1573f025cf215351e3592"}
Sep 11 23:11:49.768267 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:49.768037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"]
Sep 11 23:11:49.771402 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:11:49.771370    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa15d84_573a_4116_b92f_814de5f0b758.slice/crio-bbe57e3e17b58edf03c8e8af113c84354130d7c1d87a2e0b0f6aa34e113840a3 WatchSource:0}: Error finding container bbe57e3e17b58edf03c8e8af113c84354130d7c1d87a2e0b0f6aa34e113840a3: Status 404 returned error can't find the container with id bbe57e3e17b58edf03c8e8af113c84354130d7c1d87a2e0b0f6aa34e113840a3
Sep 11 23:11:50.178842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:50.178805    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:50.516124 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:50.516044    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw" event={"ID":"eaa15d84-573a-4116-b92f-814de5f0b758","Type":"ContainerStarted","Data":"bbe57e3e17b58edf03c8e8af113c84354130d7c1d87a2e0b0f6aa34e113840a3"}
Sep 11 23:11:52.527040 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:52.527001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w" event={"ID":"2d6d34ee-6d2c-4771-9874-7784fd39dd3f","Type":"ContainerStarted","Data":"6ab34401ff5c92f72e4b5bd300f6d4200b99de496a415d1f700ffdc145baf6c3"}
Sep 11 23:11:52.545550 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:52.545493    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c75bd8f97-b9d4w" podStartSLOduration=2.9362640129999997 podStartE2EDuration="5.545474191s" totalImagesPullingTime="2.609210178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:11:52.543827828 +0000 UTC m=+627.755799002" watchObservedRunningTime="2026-09-11 23:11:52.545474191 +0000 UTC m=+627.757445368"
Sep 11 23:11:53.343107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:53.343069    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 23:11:53.343178 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:53.343141    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 23:11:53.532650 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:53.532611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw" event={"ID":"eaa15d84-573a-4116-b92f-814de5f0b758","Type":"ContainerStarted","Data":"81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662"}
Sep 11 23:11:53.555343 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:53.555287    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw" podStartSLOduration=0.986970924 podStartE2EDuration="4.555266184s" totalImagesPullingTime="3.56829526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:11:53.554583189 +0000 UTC m=+628.766554363" watchObservedRunningTime="2026-09-11 23:11:53.555266184 +0000 UTC m=+628.767237359"
Sep 11 23:11:54.118627 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:54.118584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:54.227630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:54.227593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:54.536614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:54.536542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:54.538327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:54.538302    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:11:54.609522 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:54.609487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:54.799687 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:54.799614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:54.901988 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:54.901959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:55.385832 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.385801    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:55.457131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.457099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:55.515725 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.515688    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-dbcccc57d-flhsf" podUID="569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" containerName="console" containerID="cri-o://c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f" gracePeriod=15
Sep 11 23:11:55.761803 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.761775    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:11:55.785010 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.784981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qhsq\" (UniqueName: \"kubernetes.io/projected/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-kube-api-access-5qhsq\") pod \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") "
Sep 11 23:11:55.785145 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-oauth-serving-cert\") pod \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") "
Sep 11 23:11:55.785145 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-service-ca\") pod \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") "
Sep 11 23:11:55.785145 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785111    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-serving-cert\") pod \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") "
Sep 11 23:11:55.785318 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785282    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-trusted-ca-bundle\") pod \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") "
Sep 11 23:11:55.785380 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785363    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-config\") pod \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") "
Sep 11 23:11:55.785440 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785399    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-oauth-config\") pod \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\" (UID: \"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2\") "
Sep 11 23:11:55.785548 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785453    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-service-ca" (OuterVolumeSpecName: "service-ca") pod "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" (UID: "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:11:55.785647 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785546    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" (UID: "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:11:55.785704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785643    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:11:55.785704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785649    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" (UID: "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:11:55.785806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.785705    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-config" (OuterVolumeSpecName: "console-config") pod "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" (UID: "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:11:55.787257 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.787233    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" (UID: "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:11:55.787491 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.787472    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-kube-api-access-5qhsq" (OuterVolumeSpecName: "kube-api-access-5qhsq") pod "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" (UID: "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2"). InnerVolumeSpecName "kube-api-access-5qhsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:11:55.787783 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.787767    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" (UID: "569cddb9-a33d-49cb-a14d-4c68d6c6cbe2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:11:55.886329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.886266    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qhsq\" (UniqueName: \"kubernetes.io/projected/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-kube-api-access-5qhsq\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:11:55.886329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.886290    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-oauth-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:11:55.886329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.886302    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-serving-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:11:55.886329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.886310    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-trusted-ca-bundle\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:11:55.886329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.886319    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:11:55.886329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:55.886327    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2-console-oauth-config\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:11:56.544839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.544800    2580 generic.go:356] "Generic (PLEG): container finished" podID="569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" containerID="c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f" exitCode=0
Sep 11 23:11:56.545109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.544849    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcccc57d-flhsf"
Sep 11 23:11:56.545109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.544891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcccc57d-flhsf" event={"ID":"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2","Type":"ContainerDied","Data":"c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f"}
Sep 11 23:11:56.545109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.544968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcccc57d-flhsf" event={"ID":"569cddb9-a33d-49cb-a14d-4c68d6c6cbe2","Type":"ContainerDied","Data":"acb21622559d03b016586e79eeb86e8bc60d029186b29d706e270a882ad6c6ec"}
Sep 11 23:11:56.545109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.544996    2580 scope.go:117] "RemoveContainer" containerID="c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f"
Sep 11 23:11:56.553836 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.553817    2580 scope.go:117] "RemoveContainer" containerID="c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f"
Sep 11 23:11:56.554130 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:11:56.554104    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f\": container with ID starting with c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f not found: ID does not exist" containerID="c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f"
Sep 11 23:11:56.554222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.554145    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f"} err="failed to get container status \"c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f\": rpc error: code = NotFound desc = could not find container \"c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f\": container with ID starting with c4bd0dd39780dfc74ad30296ed3283a797898b3e68e634d7cb20ee04eeca083f not found: ID does not exist"
Sep 11 23:11:56.566575 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.566526    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dbcccc57d-flhsf"]
Sep 11 23:11:56.572562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:56.572542    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dbcccc57d-flhsf"]
Sep 11 23:11:57.414787 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:57.414736    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" path="/var/lib/kubelet/pods/569cddb9-a33d-49cb-a14d-4c68d6c6cbe2/volumes"
Sep 11 23:11:57.595581 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:57.595546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:57.657753 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:57.657697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:57.990302 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:57.990268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:58.133818 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:58.133777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:11:58.234582 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:11:58.234539    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:03.513666 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:03.513631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:05.178618 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.178584    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"]
Sep 11 23:12:05.178993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.178826    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" containerName="console"
Sep 11 23:12:05.178993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.178837    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" containerName="console"
Sep 11 23:12:05.178993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.178884    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="569cddb9-a33d-49cb-a14d-4c68d6c6cbe2" containerName="console"
Sep 11 23:12:05.187438 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.187417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.190948 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.190926    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 23:12:05.192776 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.192554    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 23:12:05.192776 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.192600    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 23:12:05.192776 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.192611    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 23:12:05.192776 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.192666    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-smzrg\""
Sep 11 23:12:05.192776 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.192760    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:12:05.199719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.199699    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"]
Sep 11 23:12:05.259692 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.259664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/221f6034-5d10-43b4-a24b-9504d2b26b42-cert\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.259850 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.259726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/221f6034-5d10-43b4-a24b-9504d2b26b42-metrics-cert\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.259850 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.259805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/221f6034-5d10-43b4-a24b-9504d2b26b42-manager-config\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.260007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.259856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpdm8\" (UniqueName: \"kubernetes.io/projected/221f6034-5d10-43b4-a24b-9504d2b26b42-kube-api-access-zpdm8\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.361041 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.361004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/221f6034-5d10-43b4-a24b-9504d2b26b42-manager-config\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.361217 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.361068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpdm8\" (UniqueName: \"kubernetes.io/projected/221f6034-5d10-43b4-a24b-9504d2b26b42-kube-api-access-zpdm8\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.361217 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.361097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/221f6034-5d10-43b4-a24b-9504d2b26b42-cert\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.361217 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.361144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/221f6034-5d10-43b4-a24b-9504d2b26b42-metrics-cert\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.361703 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.361677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/221f6034-5d10-43b4-a24b-9504d2b26b42-manager-config\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.363605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.363582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/221f6034-5d10-43b4-a24b-9504d2b26b42-metrics-cert\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.363698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.363616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/221f6034-5d10-43b4-a24b-9504d2b26b42-cert\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.376657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.372213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpdm8\" (UniqueName: \"kubernetes.io/projected/221f6034-5d10-43b4-a24b-9504d2b26b42-kube-api-access-zpdm8\") pod \"lws-controller-manager-5956d5f5bd-qzpz8\" (UID: \"221f6034-5d10-43b4-a24b-9504d2b26b42\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.496856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.496776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:05.769884 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:05.769814    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"]
Sep 11 23:12:05.773448 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:12:05.773421    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221f6034_5d10_43b4_a24b_9504d2b26b42.slice/crio-199687b72fb0e28fd5599268430a6ecf3242170f96ff8eaf4bac7795c255b786 WatchSource:0}: Error finding container 199687b72fb0e28fd5599268430a6ecf3242170f96ff8eaf4bac7795c255b786: Status 404 returned error can't find the container with id 199687b72fb0e28fd5599268430a6ecf3242170f96ff8eaf4bac7795c255b786
Sep 11 23:12:06.586587 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:06.586549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8" event={"ID":"221f6034-5d10-43b4-a24b-9504d2b26b42","Type":"ContainerStarted","Data":"199687b72fb0e28fd5599268430a6ecf3242170f96ff8eaf4bac7795c255b786"}
Sep 11 23:12:07.781356 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:07.781293    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:07.882278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:07.882230    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:08.050949 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:08.050837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:08.305263 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:08.305175    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:08.596348 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:08.596313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8" event={"ID":"221f6034-5d10-43b4-a24b-9504d2b26b42","Type":"ContainerStarted","Data":"900f1e6a33c2ce37d151667d733f4436a087330a224968d1e0f484aca4057169"}
Sep 11 23:12:08.596522 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:08.596422    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:08.614427 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:08.614385    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8" podStartSLOduration=0.992275569 podStartE2EDuration="3.614370517s" totalImagesPullingTime="2.622094948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:12:08.613666847 +0000 UTC m=+643.825638022" watchObservedRunningTime="2026-09-11 23:12:08.614370517 +0000 UTC m=+643.826341691"
Sep 11 23:12:09.105066 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.105012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:09.207928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.207883    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"]
Sep 11 23:12:09.208101 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.208080    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw" podUID="eaa15d84-573a-4116-b92f-814de5f0b758" containerName="discovery" containerID="cri-o://81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662" gracePeriod=30
Sep 11 23:12:09.304677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.304647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:09.394509 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.394474    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:09.449129 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.449106    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:12:09.592703 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.592665    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-dns-cert\") pod \"eaa15d84-573a-4116-b92f-814de5f0b758\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") "
Sep 11 23:12:09.592703 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.592712    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaa15d84-573a-4116-b92f-814de5f0b758-local-certs\") pod \"eaa15d84-573a-4116-b92f-814de5f0b758\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") "
Sep 11 23:12:09.592959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.592737    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-kubeconfig\") pod \"eaa15d84-573a-4116-b92f-814de5f0b758\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") "
Sep 11 23:12:09.592959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.592763    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-istio-token\") pod \"eaa15d84-573a-4116-b92f-814de5f0b758\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") "
Sep 11 23:12:09.592959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.592812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzlzg\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-kube-api-access-vzlzg\") pod \"eaa15d84-573a-4116-b92f-814de5f0b758\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") "
Sep 11 23:12:09.593131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.592970    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-ca-configmap\") pod \"eaa15d84-573a-4116-b92f-814de5f0b758\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") "
Sep 11 23:12:09.593131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.593054    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-cacerts\") pod \"eaa15d84-573a-4116-b92f-814de5f0b758\" (UID: \"eaa15d84-573a-4116-b92f-814de5f0b758\") "
Sep 11 23:12:09.594102 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.594069    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "eaa15d84-573a-4116-b92f-814de5f0b758" (UID: "eaa15d84-573a-4116-b92f-814de5f0b758"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:12:09.595628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.595582    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "eaa15d84-573a-4116-b92f-814de5f0b758" (UID: "eaa15d84-573a-4116-b92f-814de5f0b758"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:12:09.595736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.595614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "eaa15d84-573a-4116-b92f-814de5f0b758" (UID: "eaa15d84-573a-4116-b92f-814de5f0b758"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:12:09.595736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.595617    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-kube-api-access-vzlzg" (OuterVolumeSpecName: "kube-api-access-vzlzg") pod "eaa15d84-573a-4116-b92f-814de5f0b758" (UID: "eaa15d84-573a-4116-b92f-814de5f0b758"). InnerVolumeSpecName "kube-api-access-vzlzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:12:09.596031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.596002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaa15d84-573a-4116-b92f-814de5f0b758-local-certs" (OuterVolumeSpecName: "local-certs") pod "eaa15d84-573a-4116-b92f-814de5f0b758" (UID: "eaa15d84-573a-4116-b92f-814de5f0b758"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:12:09.596274 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.596251    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-cacerts" (OuterVolumeSpecName: "cacerts") pod "eaa15d84-573a-4116-b92f-814de5f0b758" (UID: "eaa15d84-573a-4116-b92f-814de5f0b758"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:12:09.596357 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.596273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-istio-token" (OuterVolumeSpecName: "istio-token") pod "eaa15d84-573a-4116-b92f-814de5f0b758" (UID: "eaa15d84-573a-4116-b92f-814de5f0b758"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:12:09.606168 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.606139    2580 generic.go:356] "Generic (PLEG): container finished" podID="eaa15d84-573a-4116-b92f-814de5f0b758" containerID="81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662" exitCode=0
Sep 11 23:12:09.606289 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.606201    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"
Sep 11 23:12:09.606289 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.606222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw" event={"ID":"eaa15d84-573a-4116-b92f-814de5f0b758","Type":"ContainerDied","Data":"81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662"}
Sep 11 23:12:09.606403 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.606336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw" event={"ID":"eaa15d84-573a-4116-b92f-814de5f0b758","Type":"ContainerDied","Data":"bbe57e3e17b58edf03c8e8af113c84354130d7c1d87a2e0b0f6aa34e113840a3"}
Sep 11 23:12:09.606403 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.606381    2580 scope.go:117] "RemoveContainer" containerID="81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662"
Sep 11 23:12:09.614434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.614415    2580 scope.go:117] "RemoveContainer" containerID="81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662"
Sep 11 23:12:09.614696 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:12:09.614676    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662\": container with ID starting with 81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662 not found: ID does not exist" containerID="81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662"
Sep 11 23:12:09.614795 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.614703    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662"} err="failed to get container status \"81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662\": rpc error: code = NotFound desc = could not find container \"81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662\": container with ID starting with 81f14d3d3e1da4aa82b8e5c455e41d76efe5050d18ddfa723acdf51c3c1c4662 not found: ID does not exist"
Sep 11 23:12:09.628666 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.628618    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"]
Sep 11 23:12:09.632555 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.632526    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4fsw"]
Sep 11 23:12:09.693952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.693901    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-cacerts\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:12:09.693952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.693949    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-dns-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:12:09.694164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.693965    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaa15d84-573a-4116-b92f-814de5f0b758-local-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:12:09.694164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.693978    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eaa15d84-573a-4116-b92f-814de5f0b758-istio-kubeconfig\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:12:09.694164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.693991    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-istio-token\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:12:09.694164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.694003    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzlzg\" (UniqueName: \"kubernetes.io/projected/eaa15d84-573a-4116-b92f-814de5f0b758-kube-api-access-vzlzg\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:12:09.694164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:09.694016    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eaa15d84-573a-4116-b92f-814de5f0b758-istio-csr-ca-configmap\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:12:11.413052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:11.413015    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaa15d84-573a-4116-b92f-814de5f0b758" path="/var/lib/kubelet/pods/eaa15d84-573a-4116-b92f-814de5f0b758/volumes"
Sep 11 23:12:19.608957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:19.608905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-qzpz8"
Sep 11 23:12:58.671203 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:58.671164    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:12:58.788307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:12:58.788267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:14.083538 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:14.083507    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:14.918234 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:14.918202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:15.680311 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:15.680276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:18.828888 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:18.828799    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:18.922129 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:18.922092    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:19.287564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:19.287488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:20.331457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:20.331426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:21.888547 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:21.888516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:22.579756 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:22.579720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:24.131550 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:24.131518    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:25.134096 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:25.134059    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:26.269226 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.269187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:26.278052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.278026    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"]
Sep 11 23:13:26.278278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.278267    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaa15d84-573a-4116-b92f-814de5f0b758" containerName="discovery"
Sep 11 23:13:26.278320 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.278280    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaa15d84-573a-4116-b92f-814de5f0b758" containerName="discovery"
Sep 11 23:13:26.278354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.278330    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaa15d84-573a-4116-b92f-814de5f0b758" containerName="discovery"
Sep 11 23:13:26.281021 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.281004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.285602 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.285582    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 23:13:26.285797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.285631    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mzkwj\""
Sep 11 23:13:26.286096 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.286078    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 23:13:26.286866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.286849    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 23:13:26.286866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.286853    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 23:13:26.290984 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.290962    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"]
Sep 11 23:13:26.368473 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.368446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.368636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.368477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsznz\" (UniqueName: \"kubernetes.io/projected/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-kube-api-access-gsznz\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.368636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.368515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.369741 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.369717    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:26.469091 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.469053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.469248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.469137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.469248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.469157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsznz\" (UniqueName: \"kubernetes.io/projected/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-kube-api-access-gsznz\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.469851 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.469831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.471551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.471532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.478456 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.478434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsznz\" (UniqueName: \"kubernetes.io/projected/eeebb0b9-3e95-4c85-a6d9-05acb01b84db-kube-api-access-gsznz\") pod \"kuadrant-console-plugin-fbdd4f445-br7hd\" (UID: \"eeebb0b9-3e95-4c85-a6d9-05acb01b84db\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.590296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.590255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"
Sep 11 23:13:26.867262 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.867177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd"]
Sep 11 23:13:26.870140 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:13:26.870112    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeeebb0b9_3e95_4c85_a6d9_05acb01b84db.slice/crio-72c366986f9278ab7dc7d9a59201e82617172c6c73e10b3477cb58a600c1105d WatchSource:0}: Error finding container 72c366986f9278ab7dc7d9a59201e82617172c6c73e10b3477cb58a600c1105d: Status 404 returned error can't find the container with id 72c366986f9278ab7dc7d9a59201e82617172c6c73e10b3477cb58a600c1105d
Sep 11 23:13:26.888263 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:26.888237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd" event={"ID":"eeebb0b9-3e95-4c85-a6d9-05acb01b84db","Type":"ContainerStarted","Data":"72c366986f9278ab7dc7d9a59201e82617172c6c73e10b3477cb58a600c1105d"}
Sep 11 23:13:29.291301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:29.291264    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:29.346683 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:29.346645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:29.694227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:29.694185    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:29.795516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:29.795430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:30.107228 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:30.107171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:30.207035 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:30.207000    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:31.911396 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:31.911364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd" event={"ID":"eeebb0b9-3e95-4c85-a6d9-05acb01b84db","Type":"ContainerStarted","Data":"a71c6b7a23d7502326309664d2fde1098460dfb065b9ad20849f480ca15349dc"}
Sep 11 23:13:31.938028 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:31.937907    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-br7hd" podStartSLOduration=1.256602768 podStartE2EDuration="5.937889146s" totalImagesPullingTime="4.681286378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:13:31.937280558 +0000 UTC m=+727.149251732" watchObservedRunningTime="2026-09-11 23:13:31.937889146 +0000 UTC m=+727.149860320"
Sep 11 23:13:38.116152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:38.116123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:38.217865 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:38.217824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:41.108111 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:41.108073    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:41.212075 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:41.212044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:43.542827 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:43.542793    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:43.645459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:43.645426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:50.502592 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:50.502551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:50.564823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:50.564788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:50.967329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:50.967294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:51.024049 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:51.024007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:51.101906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:51.101850    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:51.202529 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:51.202493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:13:59.920512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:13:59.920473    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:00.018790 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:00.018756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:02.924823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:02.924773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:03.023564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:03.023514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:05.374405 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:05.374369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:05.443689 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:05.443647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:08.520497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:08.520466    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:08.933715 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:08.933686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:09.040531 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:09.040500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:09.247630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:09.247529    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:24.608224 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:24.608184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:24.706045 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:24.706013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:24.763627 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:24.763584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:25.401203 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.401170    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j69lf"]
Sep 11 23:14:25.404202 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.404185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf"
Sep 11 23:14:25.407131 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.407106    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pr8sg\""
Sep 11 23:14:25.420298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.420256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j69lf"]
Sep 11 23:14:25.440102 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.440066    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:25.520605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.520569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grjhk\" (UniqueName: \"kubernetes.io/projected/6db531a7-8bdc-433f-8728-04850817260f-kube-api-access-grjhk\") pod \"authorino-7bcfd8f9cd-j69lf\" (UID: \"6db531a7-8bdc-433f-8728-04850817260f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf"
Sep 11 23:14:25.522570 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.522536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:25.523128 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.523107    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-22cmb"]
Sep 11 23:14:25.526130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.526113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-22cmb"
Sep 11 23:14:25.534527 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.534503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-22cmb"]
Sep 11 23:14:25.621606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.621563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-496kh\" (UniqueName: \"kubernetes.io/projected/fbe2ad05-1286-4025-99cd-6e9b0db62360-kube-api-access-496kh\") pod \"authorino-59c974bd48-22cmb\" (UID: \"fbe2ad05-1286-4025-99cd-6e9b0db62360\") " pod="kuadrant-system/authorino-59c974bd48-22cmb"
Sep 11 23:14:25.622040 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.621633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grjhk\" (UniqueName: \"kubernetes.io/projected/6db531a7-8bdc-433f-8728-04850817260f-kube-api-access-grjhk\") pod \"authorino-7bcfd8f9cd-j69lf\" (UID: \"6db531a7-8bdc-433f-8728-04850817260f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf"
Sep 11 23:14:25.629633 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.629605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grjhk\" (UniqueName: \"kubernetes.io/projected/6db531a7-8bdc-433f-8728-04850817260f-kube-api-access-grjhk\") pod \"authorino-7bcfd8f9cd-j69lf\" (UID: \"6db531a7-8bdc-433f-8728-04850817260f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf"
Sep 11 23:14:25.714870 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.714775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf"
Sep 11 23:14:25.722893 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.722872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-496kh\" (UniqueName: \"kubernetes.io/projected/fbe2ad05-1286-4025-99cd-6e9b0db62360-kube-api-access-496kh\") pod \"authorino-59c974bd48-22cmb\" (UID: \"fbe2ad05-1286-4025-99cd-6e9b0db62360\") " pod="kuadrant-system/authorino-59c974bd48-22cmb"
Sep 11 23:14:25.732233 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.732205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-496kh\" (UniqueName: \"kubernetes.io/projected/fbe2ad05-1286-4025-99cd-6e9b0db62360-kube-api-access-496kh\") pod \"authorino-59c974bd48-22cmb\" (UID: \"fbe2ad05-1286-4025-99cd-6e9b0db62360\") " pod="kuadrant-system/authorino-59c974bd48-22cmb"
Sep 11 23:14:25.835552 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.835517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-22cmb"
Sep 11 23:14:25.981349 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:25.981219    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j69lf"]
Sep 11 23:14:25.983761 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:14:25.983738    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6db531a7_8bdc_433f_8728_04850817260f.slice/crio-9da5223857e4ba0dc855467352cc4fb491a6dc5ec18cc4012c953f43c6d60a9e WatchSource:0}: Error finding container 9da5223857e4ba0dc855467352cc4fb491a6dc5ec18cc4012c953f43c6d60a9e: Status 404 returned error can't find the container with id 9da5223857e4ba0dc855467352cc4fb491a6dc5ec18cc4012c953f43c6d60a9e
Sep 11 23:14:26.102091 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:26.102063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-22cmb"]
Sep 11 23:14:26.104487 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:14:26.104461    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbe2ad05_1286_4025_99cd_6e9b0db62360.slice/crio-2bb2a596a118ca67f1d4b85bfa1ad72f3cfd3ae6e2ab2a949bfaa394f6d01c9e WatchSource:0}: Error finding container 2bb2a596a118ca67f1d4b85bfa1ad72f3cfd3ae6e2ab2a949bfaa394f6d01c9e: Status 404 returned error can't find the container with id 2bb2a596a118ca67f1d4b85bfa1ad72f3cfd3ae6e2ab2a949bfaa394f6d01c9e
Sep 11 23:14:26.110268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:26.110241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-22cmb" event={"ID":"fbe2ad05-1286-4025-99cd-6e9b0db62360","Type":"ContainerStarted","Data":"2bb2a596a118ca67f1d4b85bfa1ad72f3cfd3ae6e2ab2a949bfaa394f6d01c9e"}
Sep 11 23:14:26.111462 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:26.111439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf" event={"ID":"6db531a7-8bdc-433f-8728-04850817260f","Type":"ContainerStarted","Data":"9da5223857e4ba0dc855467352cc4fb491a6dc5ec18cc4012c953f43c6d60a9e"}
Sep 11 23:14:29.129296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:29.129260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-22cmb" event={"ID":"fbe2ad05-1286-4025-99cd-6e9b0db62360","Type":"ContainerStarted","Data":"077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d"}
Sep 11 23:14:29.130700 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:29.130675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf" event={"ID":"6db531a7-8bdc-433f-8728-04850817260f","Type":"ContainerStarted","Data":"3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952"}
Sep 11 23:14:29.145379 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:29.145328    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-22cmb" podStartSLOduration=1.593079023 podStartE2EDuration="4.145313215s" totalImagesPullingTime="2.552234192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:14:29.144671441 +0000 UTC m=+784.356642615" watchObservedRunningTime="2026-09-11 23:14:29.145313215 +0000 UTC m=+784.357284434"
Sep 11 23:14:29.161284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:29.161233    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf" podStartSLOduration=1.499029739 podStartE2EDuration="4.161217707s" totalImagesPullingTime="2.662187968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:14:29.160225522 +0000 UTC m=+784.372196697" watchObservedRunningTime="2026-09-11 23:14:29.161217707 +0000 UTC m=+784.373188880"
Sep 11 23:14:29.183073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:29.183040    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j69lf"]
Sep 11 23:14:31.138813 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:31.138762    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf" podUID="6db531a7-8bdc-433f-8728-04850817260f" containerName="authorino" containerID="cri-o://3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952" gracePeriod=30
Sep 11 23:14:31.386018 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:31.385995    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf"
Sep 11 23:14:31.472678 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:31.472587    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grjhk\" (UniqueName: \"kubernetes.io/projected/6db531a7-8bdc-433f-8728-04850817260f-kube-api-access-grjhk\") pod \"6db531a7-8bdc-433f-8728-04850817260f\" (UID: \"6db531a7-8bdc-433f-8728-04850817260f\") "
Sep 11 23:14:31.474761 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:31.474729    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db531a7-8bdc-433f-8728-04850817260f-kube-api-access-grjhk" (OuterVolumeSpecName: "kube-api-access-grjhk") pod "6db531a7-8bdc-433f-8728-04850817260f" (UID: "6db531a7-8bdc-433f-8728-04850817260f"). InnerVolumeSpecName "kube-api-access-grjhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:14:31.573381 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:31.573351    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grjhk\" (UniqueName: \"kubernetes.io/projected/6db531a7-8bdc-433f-8728-04850817260f-kube-api-access-grjhk\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:14:32.144190 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.144153    2580 generic.go:356] "Generic (PLEG): container finished" podID="6db531a7-8bdc-433f-8728-04850817260f" containerID="3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952" exitCode=0
Sep 11 23:14:32.144190 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.144193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf" event={"ID":"6db531a7-8bdc-433f-8728-04850817260f","Type":"ContainerDied","Data":"3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952"}
Sep 11 23:14:32.144731 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.144195    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf"
Sep 11 23:14:32.144731 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.144224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j69lf" event={"ID":"6db531a7-8bdc-433f-8728-04850817260f","Type":"ContainerDied","Data":"9da5223857e4ba0dc855467352cc4fb491a6dc5ec18cc4012c953f43c6d60a9e"}
Sep 11 23:14:32.144731 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.144243    2580 scope.go:117] "RemoveContainer" containerID="3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952"
Sep 11 23:14:32.152730 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.152690    2580 scope.go:117] "RemoveContainer" containerID="3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952"
Sep 11 23:14:32.153040 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:14:32.153016    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952\": container with ID starting with 3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952 not found: ID does not exist" containerID="3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952"
Sep 11 23:14:32.153136 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.153045    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952"} err="failed to get container status \"3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952\": rpc error: code = NotFound desc = could not find container \"3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952\": container with ID starting with 3b4c52a19f0362a63abbd7769fcad6df8fcd45ccc1d48bad1686393368882952 not found: ID does not exist"
Sep 11 23:14:32.166972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.166954    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j69lf"]
Sep 11 23:14:32.171445 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:32.171426    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j69lf"]
Sep 11 23:14:33.412840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:33.412805    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6db531a7-8bdc-433f-8728-04850817260f" path="/var/lib/kubelet/pods/6db531a7-8bdc-433f-8728-04850817260f/volumes"
Sep 11 23:14:42.209183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:42.209136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:42.311478 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:42.311445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:42.678623 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:42.678587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:55.200132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.200052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:55.299521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.299480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:14:55.422192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.422160    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-tdzk8"]
Sep 11 23:14:55.422437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.422424    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6db531a7-8bdc-433f-8728-04850817260f" containerName="authorino"
Sep 11 23:14:55.422480 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.422439    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6db531a7-8bdc-433f-8728-04850817260f" containerName="authorino"
Sep 11 23:14:55.422513 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.422489    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6db531a7-8bdc-433f-8728-04850817260f" containerName="authorino"
Sep 11 23:14:55.425171 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.425154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tdzk8"
Sep 11 23:14:55.433427 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.433400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-tdzk8"]
Sep 11 23:14:55.519991 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.519898    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-tdzk8"]
Sep 11 23:14:55.520150 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:14:55.520132    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9jw76], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-tdzk8" podUID="65a272b7-590e-46ac-a17a-e80dae56f6da"
Sep 11 23:14:55.546056 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.546027    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-z5784"]
Sep 11 23:14:55.549167 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.549146    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:55.551817 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.551796    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 23:14:55.554237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.554214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jw76\" (UniqueName: \"kubernetes.io/projected/65a272b7-590e-46ac-a17a-e80dae56f6da-kube-api-access-9jw76\") pod \"authorino-557f876f69-tdzk8\" (UID: \"65a272b7-590e-46ac-a17a-e80dae56f6da\") " pod="kuadrant-system/authorino-557f876f69-tdzk8"
Sep 11 23:14:55.558432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.558413    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-z5784"]
Sep 11 23:14:55.639880 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.639849    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-z5784"]
Sep 11 23:14:55.640069 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:14:55.640050    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wfl9d tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-z5784" podUID="917023e4-4a9b-4a59-9e3c-242eb8ad827e"
Sep 11 23:14:55.655022 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.654995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfl9d\" (UniqueName: \"kubernetes.io/projected/917023e4-4a9b-4a59-9e3c-242eb8ad827e-kube-api-access-wfl9d\") pod \"authorino-6d4dfc649f-z5784\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") " pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:55.655109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.655025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/917023e4-4a9b-4a59-9e3c-242eb8ad827e-tls-cert\") pod \"authorino-6d4dfc649f-z5784\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") " pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:55.655109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.655076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jw76\" (UniqueName: \"kubernetes.io/projected/65a272b7-590e-46ac-a17a-e80dae56f6da-kube-api-access-9jw76\") pod \"authorino-557f876f69-tdzk8\" (UID: \"65a272b7-590e-46ac-a17a-e80dae56f6da\") " pod="kuadrant-system/authorino-557f876f69-tdzk8"
Sep 11 23:14:55.664418 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.664399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jw76\" (UniqueName: \"kubernetes.io/projected/65a272b7-590e-46ac-a17a-e80dae56f6da-kube-api-access-9jw76\") pod \"authorino-557f876f69-tdzk8\" (UID: \"65a272b7-590e-46ac-a17a-e80dae56f6da\") " pod="kuadrant-system/authorino-557f876f69-tdzk8"
Sep 11 23:14:55.755673 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.755646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfl9d\" (UniqueName: \"kubernetes.io/projected/917023e4-4a9b-4a59-9e3c-242eb8ad827e-kube-api-access-wfl9d\") pod \"authorino-6d4dfc649f-z5784\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") " pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:55.755795 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.755677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/917023e4-4a9b-4a59-9e3c-242eb8ad827e-tls-cert\") pod \"authorino-6d4dfc649f-z5784\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") " pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:55.757955 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.757929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/917023e4-4a9b-4a59-9e3c-242eb8ad827e-tls-cert\") pod \"authorino-6d4dfc649f-z5784\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") " pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:55.764631 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:55.764614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfl9d\" (UniqueName: \"kubernetes.io/projected/917023e4-4a9b-4a59-9e3c-242eb8ad827e-kube-api-access-wfl9d\") pod \"authorino-6d4dfc649f-z5784\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") " pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:56.241904 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.241874    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:56.242290 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.241886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tdzk8"
Sep 11 23:14:56.248866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.248844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:56.251486 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.251464    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tdzk8"
Sep 11 23:14:56.359930 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.359895    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/917023e4-4a9b-4a59-9e3c-242eb8ad827e-tls-cert\") pod \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") "
Sep 11 23:14:56.360084 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.359975    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jw76\" (UniqueName: \"kubernetes.io/projected/65a272b7-590e-46ac-a17a-e80dae56f6da-kube-api-access-9jw76\") pod \"65a272b7-590e-46ac-a17a-e80dae56f6da\" (UID: \"65a272b7-590e-46ac-a17a-e80dae56f6da\") "
Sep 11 23:14:56.360084 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.359997    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfl9d\" (UniqueName: \"kubernetes.io/projected/917023e4-4a9b-4a59-9e3c-242eb8ad827e-kube-api-access-wfl9d\") pod \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\" (UID: \"917023e4-4a9b-4a59-9e3c-242eb8ad827e\") "
Sep 11 23:14:56.362079 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.362042    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a272b7-590e-46ac-a17a-e80dae56f6da-kube-api-access-9jw76" (OuterVolumeSpecName: "kube-api-access-9jw76") pod "65a272b7-590e-46ac-a17a-e80dae56f6da" (UID: "65a272b7-590e-46ac-a17a-e80dae56f6da"). InnerVolumeSpecName "kube-api-access-9jw76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:14:56.362079 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.362065    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/917023e4-4a9b-4a59-9e3c-242eb8ad827e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "917023e4-4a9b-4a59-9e3c-242eb8ad827e" (UID: "917023e4-4a9b-4a59-9e3c-242eb8ad827e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:14:56.362217 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.362106    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/917023e4-4a9b-4a59-9e3c-242eb8ad827e-kube-api-access-wfl9d" (OuterVolumeSpecName: "kube-api-access-wfl9d") pod "917023e4-4a9b-4a59-9e3c-242eb8ad827e" (UID: "917023e4-4a9b-4a59-9e3c-242eb8ad827e"). InnerVolumeSpecName "kube-api-access-wfl9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:14:56.461231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.461185    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jw76\" (UniqueName: \"kubernetes.io/projected/65a272b7-590e-46ac-a17a-e80dae56f6da-kube-api-access-9jw76\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:14:56.461231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.461227    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfl9d\" (UniqueName: \"kubernetes.io/projected/917023e4-4a9b-4a59-9e3c-242eb8ad827e-kube-api-access-wfl9d\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:14:56.461231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:56.461238    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/917023e4-4a9b-4a59-9e3c-242eb8ad827e-tls-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:14:57.246139 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.246101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-tdzk8"
Sep 11 23:14:57.246580 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.246101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-z5784"
Sep 11 23:14:57.279035 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.279008    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-tdzk8"]
Sep 11 23:14:57.284873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.284846    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-tdzk8"]
Sep 11 23:14:57.306360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.306334    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-z5784"]
Sep 11 23:14:57.308163 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.308139    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-z5784"]
Sep 11 23:14:57.412259 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.412225    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65a272b7-590e-46ac-a17a-e80dae56f6da" path="/var/lib/kubelet/pods/65a272b7-590e-46ac-a17a-e80dae56f6da/volumes"
Sep 11 23:14:57.412531 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.412513    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="917023e4-4a9b-4a59-9e3c-242eb8ad827e" path="/var/lib/kubelet/pods/917023e4-4a9b-4a59-9e3c-242eb8ad827e/volumes"
Sep 11 23:14:57.938976 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.938942    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-22cmb"]
Sep 11 23:14:57.939138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:57.939105    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-22cmb" podUID="fbe2ad05-1286-4025-99cd-6e9b0db62360" containerName="authorino" containerID="cri-o://077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d" gracePeriod=30
Sep 11 23:14:58.177291 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.177269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-22cmb"
Sep 11 23:14:58.251350 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.251286    2580 generic.go:356] "Generic (PLEG): container finished" podID="fbe2ad05-1286-4025-99cd-6e9b0db62360" containerID="077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d" exitCode=0
Sep 11 23:14:58.251350 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.251323    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-22cmb"
Sep 11 23:14:58.251727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.251368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-22cmb" event={"ID":"fbe2ad05-1286-4025-99cd-6e9b0db62360","Type":"ContainerDied","Data":"077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d"}
Sep 11 23:14:58.251727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.251407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-22cmb" event={"ID":"fbe2ad05-1286-4025-99cd-6e9b0db62360","Type":"ContainerDied","Data":"2bb2a596a118ca67f1d4b85bfa1ad72f3cfd3ae6e2ab2a949bfaa394f6d01c9e"}
Sep 11 23:14:58.251727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.251428    2580 scope.go:117] "RemoveContainer" containerID="077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d"
Sep 11 23:14:58.258713 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.258698    2580 scope.go:117] "RemoveContainer" containerID="077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d"
Sep 11 23:14:58.258996 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:14:58.258970    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d\": container with ID starting with 077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d not found: ID does not exist" containerID="077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d"
Sep 11 23:14:58.259089 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.259005    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d"} err="failed to get container status \"077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d\": rpc error: code = NotFound desc = could not find container \"077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d\": container with ID starting with 077c2d34629df25ac7f04bb883a8373a27a01c6e3410148c3fb994c9b7bb5d1d not found: ID does not exist"
Sep 11 23:14:58.274604 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.274588    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-496kh\" (UniqueName: \"kubernetes.io/projected/fbe2ad05-1286-4025-99cd-6e9b0db62360-kube-api-access-496kh\") pod \"fbe2ad05-1286-4025-99cd-6e9b0db62360\" (UID: \"fbe2ad05-1286-4025-99cd-6e9b0db62360\") "
Sep 11 23:14:58.276374 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.276351    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe2ad05-1286-4025-99cd-6e9b0db62360-kube-api-access-496kh" (OuterVolumeSpecName: "kube-api-access-496kh") pod "fbe2ad05-1286-4025-99cd-6e9b0db62360" (UID: "fbe2ad05-1286-4025-99cd-6e9b0db62360"). InnerVolumeSpecName "kube-api-access-496kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:14:58.376018 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.375991    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-496kh\" (UniqueName: \"kubernetes.io/projected/fbe2ad05-1286-4025-99cd-6e9b0db62360-kube-api-access-496kh\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:14:58.571431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.571404    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-22cmb"]
Sep 11 23:14:58.575115 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:58.575090    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-22cmb"]
Sep 11 23:14:59.412737 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:14:59.412705    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbe2ad05-1286-4025-99cd-6e9b0db62360" path="/var/lib/kubelet/pods/fbe2ad05-1286-4025-99cd-6e9b0db62360/volumes"
Sep 11 23:15:02.666238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.666201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:02.694645 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.694614    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f856d68c-42sbm"]
Sep 11 23:15:02.694905 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.694892    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbe2ad05-1286-4025-99cd-6e9b0db62360" containerName="authorino"
Sep 11 23:15:02.694969 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.694907    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbe2ad05-1286-4025-99cd-6e9b0db62360" containerName="authorino"
Sep 11 23:15:02.695016 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.695000    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbe2ad05-1286-4025-99cd-6e9b0db62360" containerName="authorino"
Sep 11 23:15:02.700508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.700489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.703802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.703779    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 23:15:02.703953 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.703818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 23:15:02.703953 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.703838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t2spv\""
Sep 11 23:15:02.703953 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.703898    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 23:15:02.712791 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.712476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f856d68c-42sbm"]
Sep 11 23:15:02.808307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.808279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z4l5\" (UniqueName: \"kubernetes.io/projected/3962fa76-0422-45d2-a199-877a7de5a700-kube-api-access-9z4l5\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.808437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.808319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-metrics-certs\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.808437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.808355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.809237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.809213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:02.892495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.892464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:02.908898 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.908872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z4l5\" (UniqueName: \"kubernetes.io/projected/3962fa76-0422-45d2-a199-877a7de5a700-kube-api-access-9z4l5\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.909023 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.908924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-metrics-certs\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.909023 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.908957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.909097 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:15:02.909068    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 23:15:02.909152 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:15:02.909138    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert podName:3962fa76-0422-45d2-a199-877a7de5a700 nodeName:}" failed. No retries permitted until 2026-09-11 23:15:03.409117394 +0000 UTC m=+818.621088567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert") pod "maas-controller-5f856d68c-42sbm" (UID: "3962fa76-0422-45d2-a199-877a7de5a700") : secret "maas-controller-webhook-cert" not found
Sep 11 23:15:02.911281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.911260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-metrics-certs\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:02.923313 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:02.923252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z4l5\" (UniqueName: \"kubernetes.io/projected/3962fa76-0422-45d2-a199-877a7de5a700-kube-api-access-9z4l5\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:03.027041 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:03.027014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:03.417046 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:03.417019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:03.419276 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:03.419253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert\") pod \"maas-controller-5f856d68c-42sbm\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") " pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:03.614547 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:03.614511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:03.882741 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:03.882712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f856d68c-42sbm"]
Sep 11 23:15:03.886193 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:15:03.886169    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3962fa76_0422_45d2_a199_877a7de5a700.slice/crio-373fd17fad03eb56f028328fe83b19898b5c7ccf7280655e9c452774732091a5 WatchSource:0}: Error finding container 373fd17fad03eb56f028328fe83b19898b5c7ccf7280655e9c452774732091a5: Status 404 returned error can't find the container with id 373fd17fad03eb56f028328fe83b19898b5c7ccf7280655e9c452774732091a5
Sep 11 23:15:04.278591 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:04.278507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f856d68c-42sbm" event={"ID":"3962fa76-0422-45d2-a199-877a7de5a700","Type":"ContainerStarted","Data":"373fd17fad03eb56f028328fe83b19898b5c7ccf7280655e9c452774732091a5"}
Sep 11 23:15:06.043793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:06.043753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:06.140958 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:06.140898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:06.289742 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:06.289652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f856d68c-42sbm" event={"ID":"3962fa76-0422-45d2-a199-877a7de5a700","Type":"ContainerStarted","Data":"f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a"}
Sep 11 23:15:06.289742 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:06.289730    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:06.307734 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:06.307686    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f856d68c-42sbm" podStartSLOduration=2.197819182 podStartE2EDuration="4.307670816s" totalImagesPullingTime="2.109851634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:15:06.30648206 +0000 UTC m=+821.518453236" watchObservedRunningTime="2026-09-11 23:15:06.307670816 +0000 UTC m=+821.519641990"
Sep 11 23:15:16.999511 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:16.999477    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f856d68c-42sbm"]
Sep 11 23:15:16.999986 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:16.999706    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f856d68c-42sbm" podUID="3962fa76-0422-45d2-a199-877a7de5a700" containerName="manager" containerID="cri-o://f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a" gracePeriod=30
Sep 11 23:15:17.001546 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.001521    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:17.236367 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.236339    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:17.321140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.321108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-metrics-certs\") pod \"3962fa76-0422-45d2-a199-877a7de5a700\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") "
Sep 11 23:15:17.321300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.321158    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z4l5\" (UniqueName: \"kubernetes.io/projected/3962fa76-0422-45d2-a199-877a7de5a700-kube-api-access-9z4l5\") pod \"3962fa76-0422-45d2-a199-877a7de5a700\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") "
Sep 11 23:15:17.321300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.321183    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert\") pod \"3962fa76-0422-45d2-a199-877a7de5a700\" (UID: \"3962fa76-0422-45d2-a199-877a7de5a700\") "
Sep 11 23:15:17.323296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.323266    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3962fa76-0422-45d2-a199-877a7de5a700-kube-api-access-9z4l5" (OuterVolumeSpecName: "kube-api-access-9z4l5") pod "3962fa76-0422-45d2-a199-877a7de5a700" (UID: "3962fa76-0422-45d2-a199-877a7de5a700"). InnerVolumeSpecName "kube-api-access-9z4l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:15:17.323407 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.323298    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3962fa76-0422-45d2-a199-877a7de5a700" (UID: "3962fa76-0422-45d2-a199-877a7de5a700"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:15:17.323407 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.323369    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3962fa76-0422-45d2-a199-877a7de5a700" (UID: "3962fa76-0422-45d2-a199-877a7de5a700"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:15:17.336056 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.336033    2580 generic.go:356] "Generic (PLEG): container finished" podID="3962fa76-0422-45d2-a199-877a7de5a700" containerID="f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a" exitCode=0
Sep 11 23:15:17.336156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.336078    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f856d68c-42sbm"
Sep 11 23:15:17.336156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.336088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f856d68c-42sbm" event={"ID":"3962fa76-0422-45d2-a199-877a7de5a700","Type":"ContainerDied","Data":"f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a"}
Sep 11 23:15:17.336156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.336135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f856d68c-42sbm" event={"ID":"3962fa76-0422-45d2-a199-877a7de5a700","Type":"ContainerDied","Data":"373fd17fad03eb56f028328fe83b19898b5c7ccf7280655e9c452774732091a5"}
Sep 11 23:15:17.336269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.336157    2580 scope.go:117] "RemoveContainer" containerID="f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a"
Sep 11 23:15:17.344207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.344192    2580 scope.go:117] "RemoveContainer" containerID="f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a"
Sep 11 23:15:17.344480 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:15:17.344462    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a\": container with ID starting with f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a not found: ID does not exist" containerID="f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a"
Sep 11 23:15:17.344532 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.344488    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a"} err="failed to get container status \"f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a\": rpc error: code = NotFound desc = could not find container \"f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a\": container with ID starting with f5e10caf57c27149c65cf33841254092c93a211d01045d78d9a0db02dbb6038a not found: ID does not exist"
Sep 11 23:15:17.358289 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.358266    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f856d68c-42sbm"]
Sep 11 23:15:17.362755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.362737    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f856d68c-42sbm"]
Sep 11 23:15:17.413254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.413224    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3962fa76-0422-45d2-a199-877a7de5a700" path="/var/lib/kubelet/pods/3962fa76-0422-45d2-a199-877a7de5a700/volumes"
Sep 11 23:15:17.422251 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.422232    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z4l5\" (UniqueName: \"kubernetes.io/projected/3962fa76-0422-45d2-a199-877a7de5a700-kube-api-access-9z4l5\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:15:17.422343 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.422254    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-webhook-cert\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:15:17.422343 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:17.422264    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3962fa76-0422-45d2-a199-877a7de5a700-metrics-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:15:37.755625 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:37.755575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:37.856378 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:37.856339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:39.217514 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:39.217470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:39.323509 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:39.323472    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:39.429224 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:39.429190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:39.539434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:39.539360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:39.939137 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:39.939097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:40.037576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.037549    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:40.064933 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.064882    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"]
Sep 11 23:15:40.065237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.065221    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3962fa76-0422-45d2-a199-877a7de5a700" containerName="manager"
Sep 11 23:15:40.065317 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.065241    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3962fa76-0422-45d2-a199-877a7de5a700" containerName="manager"
Sep 11 23:15:40.065317 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.065306    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3962fa76-0422-45d2-a199-877a7de5a700" containerName="manager"
Sep 11 23:15:40.071834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.071815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.075465 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.075445    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fss2d\""
Sep 11 23:15:40.077228 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.077209    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 23:15:40.086395 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.086362    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"]
Sep 11 23:15:40.182696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.182661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5174e8da-57b9-402d-911d-c03550f89619-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6cdvp\" (UID: \"5174e8da-57b9-402d-911d-c03550f89619\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.182843 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.182707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf4j2\" (UniqueName: \"kubernetes.io/projected/5174e8da-57b9-402d-911d-c03550f89619-kube-api-access-rf4j2\") pod \"payload-processing-5b54b7fbb7-6cdvp\" (UID: \"5174e8da-57b9-402d-911d-c03550f89619\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.284061 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.283984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5174e8da-57b9-402d-911d-c03550f89619-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6cdvp\" (UID: \"5174e8da-57b9-402d-911d-c03550f89619\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.284061 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.284027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf4j2\" (UniqueName: \"kubernetes.io/projected/5174e8da-57b9-402d-911d-c03550f89619-kube-api-access-rf4j2\") pod \"payload-processing-5b54b7fbb7-6cdvp\" (UID: \"5174e8da-57b9-402d-911d-c03550f89619\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.284682 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.284661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5174e8da-57b9-402d-911d-c03550f89619-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-6cdvp\" (UID: \"5174e8da-57b9-402d-911d-c03550f89619\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.293043 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.293021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf4j2\" (UniqueName: \"kubernetes.io/projected/5174e8da-57b9-402d-911d-c03550f89619-kube-api-access-rf4j2\") pod \"payload-processing-5b54b7fbb7-6cdvp\" (UID: \"5174e8da-57b9-402d-911d-c03550f89619\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.381623 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.381602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:40.650705 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:40.650680    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"]
Sep 11 23:15:40.653363 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:15:40.653337    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5174e8da_57b9_402d_911d_c03550f89619.slice/crio-e091b8f3f1e3e15d142384b9a9e1568a83271fc25ef91788a0383b28a21d7c91 WatchSource:0}: Error finding container e091b8f3f1e3e15d142384b9a9e1568a83271fc25ef91788a0383b28a21d7c91: Status 404 returned error can't find the container with id e091b8f3f1e3e15d142384b9a9e1568a83271fc25ef91788a0383b28a21d7c91
Sep 11 23:15:41.426786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:41.426751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp" event={"ID":"5174e8da-57b9-402d-911d-c03550f89619","Type":"ContainerStarted","Data":"e091b8f3f1e3e15d142384b9a9e1568a83271fc25ef91788a0383b28a21d7c91"}
Sep 11 23:15:42.433390 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:42.433313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp" event={"ID":"5174e8da-57b9-402d-911d-c03550f89619","Type":"ContainerStarted","Data":"814a69b56d15282b0b361dc35d962c2f9d34d1037d6ce14b7835c5156fd30a91"}
Sep 11 23:15:42.433390 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:42.433370    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:15:42.450498 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:42.450442    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp" podStartSLOduration=0.97288134 podStartE2EDuration="2.450423671s" totalImagesPullingTime="1.477542331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:15:42.449754681 +0000 UTC m=+857.661725878" watchObservedRunningTime="2026-09-11 23:15:42.450423671 +0000 UTC m=+857.662394848"
Sep 11 23:15:50.106750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:50.106697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:50.207459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:50.207424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:15:53.438755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:15:53.438726    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-6cdvp"
Sep 11 23:16:12.223185 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.223148    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"]
Sep 11 23:16:12.226275 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.226258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.229161 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.229130    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 23:16:12.230601 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.230581    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5pwjc\""
Sep 11 23:16:12.230721 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.230703    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 23:16:12.230785 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.230773    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 23:16:12.236579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.236557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"]
Sep 11 23:16:12.319539 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.319505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:12.325282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.325255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.325387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.325372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s5t7\" (UniqueName: \"kubernetes.io/projected/bc7f0911-6140-4e16-9790-a9cddf61b8c4-kube-api-access-7s5t7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.325431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.325399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.325431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.325423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.325506 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.325488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc7f0911-6140-4e16-9790-a9cddf61b8c4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.325544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.325528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s5t7\" (UniqueName: \"kubernetes.io/projected/bc7f0911-6140-4e16-9790-a9cddf61b8c4-kube-api-access-7s5t7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc7f0911-6140-4e16-9790-a9cddf61b8c4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426668 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.426775 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.426680    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.428442 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.428412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc7f0911-6140-4e16-9790-a9cddf61b8c4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.428688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.428670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc7f0911-6140-4e16-9790-a9cddf61b8c4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.434115 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.434096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s5t7\" (UniqueName: \"kubernetes.io/projected/bc7f0911-6140-4e16-9790-a9cddf61b8c4-kube-api-access-7s5t7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd\" (UID: \"bc7f0911-6140-4e16-9790-a9cddf61b8c4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.558815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.558730    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:12.828432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:12.828398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"]
Sep 11 23:16:12.831363 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:16:12.831331    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc7f0911_6140_4e16_9790_a9cddf61b8c4.slice/crio-d49345efb4aa06c134d5131fc73b6c6af039fadc2d4ecb642ca2cba3e390d62b WatchSource:0}: Error finding container d49345efb4aa06c134d5131fc73b6c6af039fadc2d4ecb642ca2cba3e390d62b: Status 404 returned error can't find the container with id d49345efb4aa06c134d5131fc73b6c6af039fadc2d4ecb642ca2cba3e390d62b
Sep 11 23:16:13.546888 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:13.546853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd" event={"ID":"bc7f0911-6140-4e16-9790-a9cddf61b8c4","Type":"ContainerStarted","Data":"d49345efb4aa06c134d5131fc73b6c6af039fadc2d4ecb642ca2cba3e390d62b"}
Sep 11 23:16:14.552486 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:14.552450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd" event={"ID":"bc7f0911-6140-4e16-9790-a9cddf61b8c4","Type":"ContainerStarted","Data":"91daf2ae9c2760a43c2188cdd3984656a4f70c23d956e48c59b31144cbaeb953"}
Sep 11 23:16:14.571413 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:14.571358    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd" podStartSLOduration=1.480099111 podStartE2EDuration="2.571342895s" totalImagesPullingTime="1.091243784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:16:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:16:14.570462749 +0000 UTC m=+889.782433922" watchObservedRunningTime="2026-09-11 23:16:14.571342895 +0000 UTC m=+889.783314068"
Sep 11 23:16:19.849714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:19.849620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:22.559643 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:22.559597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:22.560109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:22.559672    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:22.572199 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:22.572175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:22.596411 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:22.596388    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd"
Sep 11 23:16:27.400704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.400668    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"]
Sep 11 23:16:27.403974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.403956    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.406834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.406813    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 23:16:27.415465 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.415437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"]
Sep 11 23:16:27.449399 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.449363    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:27.547045 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.547016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6lmx\" (UniqueName: \"kubernetes.io/projected/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-kube-api-access-p6lmx\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.547189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.547053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.547189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.547087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.547189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.547120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.547189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.547138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.547189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.547162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6lmx\" (UniqueName: \"kubernetes.io/projected/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-kube-api-access-p6lmx\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.648890 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.648873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.650754 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.650697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.650863 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.650847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.656525 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.656498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6lmx\" (UniqueName: \"kubernetes.io/projected/c0f28059-ef18-4d76-8eb0-45d0c507d5c3-kube-api-access-p6lmx\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2\" (UID: \"c0f28059-ef18-4d76-8eb0-45d0c507d5c3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.715508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.715477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:27.984577 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.984550    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"]
Sep 11 23:16:27.986819 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:16:27.986787    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f28059_ef18_4d76_8eb0_45d0c507d5c3.slice/crio-2494b14ea392f9f0c067f08772c9de7426d417b9827fd1451b38b38fdc36787f WatchSource:0}: Error finding container 2494b14ea392f9f0c067f08772c9de7426d417b9827fd1451b38b38fdc36787f: Status 404 returned error can't find the container with id 2494b14ea392f9f0c067f08772c9de7426d417b9827fd1451b38b38fdc36787f
Sep 11 23:16:27.988190 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:27.988167    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:16:28.614402 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:28.614364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2" event={"ID":"c0f28059-ef18-4d76-8eb0-45d0c507d5c3","Type":"ContainerStarted","Data":"9694c18b3d10ed886627de06a487bb7cb62f0b94b83020ab9c9e128f035c77bc"}
Sep 11 23:16:28.614766 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:28.614441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2" event={"ID":"c0f28059-ef18-4d76-8eb0-45d0c507d5c3","Type":"ContainerStarted","Data":"2494b14ea392f9f0c067f08772c9de7426d417b9827fd1451b38b38fdc36787f"}
Sep 11 23:16:28.637537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:28.637493    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2" podStartSLOduration=1.445349685 podStartE2EDuration="1.637480173s" totalImagesPullingTime="192.130488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:16:28.635849996 +0000 UTC m=+903.847821171" watchObservedRunningTime="2026-09-11 23:16:28.637480173 +0000 UTC m=+903.849451346"
Sep 11 23:16:30.759492 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:30.759453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:35.620704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.620667    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"]
Sep 11 23:16:35.623837 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.623821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.628195 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.628166    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 23:16:35.636980 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.636956    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"]
Sep 11 23:16:35.660950 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.660921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:35.808355 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.808330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blqch\" (UniqueName: \"kubernetes.io/projected/d0e94e10-ccb0-4523-bbec-64bedb676659-kube-api-access-blqch\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.808479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.808367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.808479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.808388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.808479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.808417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.808583 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.808490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e94e10-ccb0-4523-bbec-64bedb676659-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.808583 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.808515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.909882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.909806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e94e10-ccb0-4523-bbec-64bedb676659-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.909882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.909845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.910080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.909897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqch\" (UniqueName: \"kubernetes.io/projected/d0e94e10-ccb0-4523-bbec-64bedb676659-kube-api-access-blqch\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.910080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.909959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.910080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.909986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.910080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.910013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.910357 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.910331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.910459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.910381    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.910496 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.910464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.912168 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.912151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e94e10-ccb0-4523-bbec-64bedb676659-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.912336 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.912320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e94e10-ccb0-4523-bbec-64bedb676659-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.918105 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.918080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blqch\" (UniqueName: \"kubernetes.io/projected/d0e94e10-ccb0-4523-bbec-64bedb676659-kube-api-access-blqch\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9\" (UID: \"d0e94e10-ccb0-4523-bbec-64bedb676659\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:35.933978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:35.933953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:36.181387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:36.181288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:36.201783 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:36.201756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"]
Sep 11 23:16:36.204779 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:16:36.204751    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e94e10_ccb0_4523_bbec_64bedb676659.slice/crio-12d0834141517629ab78a66c08f6ecaabaae617435b4d0a0716528e5eac788fd WatchSource:0}: Error finding container 12d0834141517629ab78a66c08f6ecaabaae617435b4d0a0716528e5eac788fd: Status 404 returned error can't find the container with id 12d0834141517629ab78a66c08f6ecaabaae617435b4d0a0716528e5eac788fd
Sep 11 23:16:36.646736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:36.646705    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9" event={"ID":"d0e94e10-ccb0-4523-bbec-64bedb676659","Type":"ContainerStarted","Data":"060bf4ed334b109790b1a69aa03ab2881754386996d94ea76c94c4b02521b8f4"}
Sep 11 23:16:36.647129 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:36.646759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9" event={"ID":"d0e94e10-ccb0-4523-bbec-64bedb676659","Type":"ContainerStarted","Data":"12d0834141517629ab78a66c08f6ecaabaae617435b4d0a0716528e5eac788fd"}
Sep 11 23:16:36.664589 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:36.664540    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9" podStartSLOduration=1.486013056 podStartE2EDuration="1.664523809s" totalImagesPullingTime="178.510753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:16:36.663990487 +0000 UTC m=+911.875961665" watchObservedRunningTime="2026-09-11 23:16:36.664523809 +0000 UTC m=+911.876494982"
Sep 11 23:16:37.716361 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:37.716328    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:37.716759 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:37.716577    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:37.728976 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:37.728952    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:38.666044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:38.666017    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2"
Sep 11 23:16:41.853136 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:41.853098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:45.420943 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:45.420893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:45.934331 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:45.934301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:45.934487 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:45.934358    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:45.946686 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:45.946655    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:46.698217 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:46.698189    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9"
Sep 11 23:16:50.226855 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.226823    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"]
Sep 11 23:16:50.231272 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.231254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.234170 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.234147    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 23:16:50.239521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.239501    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"]
Sep 11 23:16:50.316018 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.315988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.316152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.316034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw7wj\" (UniqueName: \"kubernetes.io/projected/9ebc3fab-94d0-4293-ae41-6ab5848bd940-kube-api-access-lw7wj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.316152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.316068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.316152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.316097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.316152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.316137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.316294 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.316168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ebc3fab-94d0-4293-ae41-6ab5848bd940-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.321164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.321139    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:16:50.417378 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ebc3fab-94d0-4293-ae41-6ab5848bd940-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw7wj\" (UniqueName: \"kubernetes.io/projected/9ebc3fab-94d0-4293-ae41-6ab5848bd940-kube-api-access-lw7wj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417772 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.417906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.417884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.419554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.419529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ebc3fab-94d0-4293-ae41-6ab5848bd940-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.419835 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.419819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ebc3fab-94d0-4293-ae41-6ab5848bd940-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.425280 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.425258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw7wj\" (UniqueName: \"kubernetes.io/projected/9ebc3fab-94d0-4293-ae41-6ab5848bd940-kube-api-access-lw7wj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c88np\" (UID: \"9ebc3fab-94d0-4293-ae41-6ab5848bd940\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.540796 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.540734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:16:50.814422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:50.814320    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"]
Sep 11 23:16:50.816534 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:16:50.816505    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ebc3fab_94d0_4293_ae41_6ab5848bd940.slice/crio-4cda2731fff8d324176b7c246e9e047d8ffd665c593205c00cb26b2695d582e2 WatchSource:0}: Error finding container 4cda2731fff8d324176b7c246e9e047d8ffd665c593205c00cb26b2695d582e2: Status 404 returned error can't find the container with id 4cda2731fff8d324176b7c246e9e047d8ffd665c593205c00cb26b2695d582e2
Sep 11 23:16:51.708460 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:51.708426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np" event={"ID":"9ebc3fab-94d0-4293-ae41-6ab5848bd940","Type":"ContainerStarted","Data":"0b85ec13a896f2540a95a449e94981513f3de835d0437714e4257020521e9bca"}
Sep 11 23:16:51.708460 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:51.708466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np" event={"ID":"9ebc3fab-94d0-4293-ae41-6ab5848bd940","Type":"ContainerStarted","Data":"4cda2731fff8d324176b7c246e9e047d8ffd665c593205c00cb26b2695d582e2"}
Sep 11 23:16:51.727541 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:16:51.727496    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np" podStartSLOduration=1.546835008 podStartE2EDuration="1.727479962s" totalImagesPullingTime="180.644954ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:16:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:16:51.727116892 +0000 UTC m=+926.939088076" watchObservedRunningTime="2026-09-11 23:16:51.727479962 +0000 UTC m=+926.939451137"
Sep 11 23:17:00.541732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:17:00.541693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:17:00.541732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:17:00.541738    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:17:00.554293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:17:00.554273    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:17:00.619057 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:17:00.619026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:17:00.696535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:17:00.696501    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:17:00.753995 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:17:00.753965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c88np"
Sep 11 23:18:02.025180 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.025140    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85bdb47898-8c8q6"]
Sep 11 23:18:02.028354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.028330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.031245 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.031227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 23:18:02.032830 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.032811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pr8sg\""
Sep 11 23:18:02.041656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.041634    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85bdb47898-8c8q6"]
Sep 11 23:18:02.082157 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.082114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81-tls-cert\") pod \"authorino-85bdb47898-8c8q6\" (UID: \"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81\") " pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.082337 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.082184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxgxq\" (UniqueName: \"kubernetes.io/projected/58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81-kube-api-access-kxgxq\") pod \"authorino-85bdb47898-8c8q6\" (UID: \"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81\") " pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.182599 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.182563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81-tls-cert\") pod \"authorino-85bdb47898-8c8q6\" (UID: \"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81\") " pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.182810 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.182630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxgxq\" (UniqueName: \"kubernetes.io/projected/58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81-kube-api-access-kxgxq\") pod \"authorino-85bdb47898-8c8q6\" (UID: \"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81\") " pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.185229 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.185199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81-tls-cert\") pod \"authorino-85bdb47898-8c8q6\" (UID: \"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81\") " pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.192729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.192697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxgxq\" (UniqueName: \"kubernetes.io/projected/58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81-kube-api-access-kxgxq\") pod \"authorino-85bdb47898-8c8q6\" (UID: \"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81\") " pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.337641 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.337610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85bdb47898-8c8q6"
Sep 11 23:18:02.608859 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.608834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85bdb47898-8c8q6"]
Sep 11 23:18:02.611394 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:02.611364    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58e3ab41_dd59_4d7a_b9f1_25dd13d4cd81.slice/crio-857d5a404e72196a7b8bceadd8a484440a41d90383bc18988fa97ae4a46dc263 WatchSource:0}: Error finding container 857d5a404e72196a7b8bceadd8a484440a41d90383bc18988fa97ae4a46dc263: Status 404 returned error can't find the container with id 857d5a404e72196a7b8bceadd8a484440a41d90383bc18988fa97ae4a46dc263
Sep 11 23:18:02.993207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:02.993127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bdb47898-8c8q6" event={"ID":"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81","Type":"ContainerStarted","Data":"857d5a404e72196a7b8bceadd8a484440a41d90383bc18988fa97ae4a46dc263"}
Sep 11 23:18:04.000824 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:04.000784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85bdb47898-8c8q6" event={"ID":"58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81","Type":"ContainerStarted","Data":"8c2748565b277129e8a6bba5f344c2506b3a93aeb838a9b8524b45a744d3f2b3"}
Sep 11 23:18:04.019822 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:04.019772    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85bdb47898-8c8q6" podStartSLOduration=1.077044134 podStartE2EDuration="2.019754707s" totalImagesPullingTime="942.710573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:04.018548901 +0000 UTC m=+999.230520078" watchObservedRunningTime="2026-09-11 23:18:04.019754707 +0000 UTC m=+999.231725881"
Sep 11 23:18:04.580318 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:04.580279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:04.675057 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:04.675019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:05.637880 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:05.637839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:05.738634 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:05.738589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:20.687551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:20.687512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:20.790104 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:20.790069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:20.891011 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:20.890969    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:21.020642 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:21.020534    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:21.132389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:21.132352    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:21.223575 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:21.223515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:21.413038 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:21.413007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:21.561966 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:21.561932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:21.665688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:21.665590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:22.032347 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.032263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:22.130646 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.130589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:22.451256 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.451227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:22.585073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.585047    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7616-f85a38"]
Sep 11 23:18:22.588338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.588317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:22.591101 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.591081    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8cxx9\""
Sep 11 23:18:22.593591 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.593569    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-f85a38"]
Sep 11 23:18:22.657182 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.657152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm8wh\" (UniqueName: \"kubernetes.io/projected/e4df7aba-7f26-418a-a15f-3109ca355767-kube-api-access-hm8wh\") pod \"test-curl-7616-f85a38\" (UID: \"e4df7aba-7f26-418a-a15f-3109ca355767\") " pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:22.758588 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.758528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8wh\" (UniqueName: \"kubernetes.io/projected/e4df7aba-7f26-418a-a15f-3109ca355767-kube-api-access-hm8wh\") pod \"test-curl-7616-f85a38\" (UID: \"e4df7aba-7f26-418a-a15f-3109ca355767\") " pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:22.767256 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.767231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8wh\" (UniqueName: \"kubernetes.io/projected/e4df7aba-7f26-418a-a15f-3109ca355767-kube-api-access-hm8wh\") pod \"test-curl-7616-f85a38\" (UID: \"e4df7aba-7f26-418a-a15f-3109ca355767\") " pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:22.899389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:22.899363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:23.169243 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:23.169214    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-f85a38"]
Sep 11 23:18:23.173331 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:23.173304    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4df7aba_7f26_418a_a15f_3109ca355767.slice/crio-1a29d4dbd6d540e0594a9d321b8c5ff6edf4b9dace43bac44c88b61aa89ac692 WatchSource:0}: Error finding container 1a29d4dbd6d540e0594a9d321b8c5ff6edf4b9dace43bac44c88b61aa89ac692: Status 404 returned error can't find the container with id 1a29d4dbd6d540e0594a9d321b8c5ff6edf4b9dace43bac44c88b61aa89ac692
Sep 11 23:18:24.088105 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:24.088068    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-f85a38" event={"ID":"e4df7aba-7f26-418a-a15f-3109ca355767","Type":"ContainerStarted","Data":"1a29d4dbd6d540e0594a9d321b8c5ff6edf4b9dace43bac44c88b61aa89ac692"}
Sep 11 23:18:24.359144 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:24.359060    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:24.447736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:24.447701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:24.540802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:24.540770    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:25.680031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:25.679934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:25.783509 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:25.783476    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:26.011651 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.011574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:26.099298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.099266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-f85a38" event={"ID":"e4df7aba-7f26-418a-a15f-3109ca355767","Type":"ContainerStarted","Data":"5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d"}
Sep 11 23:18:26.115840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.115795    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7616-f85a38" podStartSLOduration=1.894573472 podStartE2EDuration="4.115777076s" totalImagesPullingTime="2.221203604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:26.115675141 +0000 UTC m=+1021.327646315" watchObservedRunningTime="2026-09-11 23:18:26.115777076 +0000 UTC m=+1021.327748251"
Sep 11 23:18:26.317556 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.317514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:26.419637 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.419598    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:26.442651 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.442618    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"]
Sep 11 23:18:26.445964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.445944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.448968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.448951    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-12fb6b-dockercfg-5c4gf\""
Sep 11 23:18:26.448968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.448956    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-12fb6b\""
Sep 11 23:18:26.462957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.462906    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"]
Sep 11 23:18:26.591809 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.591741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/105e5c69-f920-4ccb-9659-140d0bf31b72-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.591809 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.591783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzmbq\" (UniqueName: \"kubernetes.io/projected/105e5c69-f920-4ccb-9659-140d0bf31b72-kube-api-access-pzmbq\") pod \"payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.692611 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.692576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzmbq\" (UniqueName: \"kubernetes.io/projected/105e5c69-f920-4ccb-9659-140d0bf31b72-kube-api-access-pzmbq\") pod \"payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.692998 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.692698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/105e5c69-f920-4ccb-9659-140d0bf31b72-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.693572 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.693551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/105e5c69-f920-4ccb-9659-140d0bf31b72-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.705523 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.705498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzmbq\" (UniqueName: \"kubernetes.io/projected/105e5c69-f920-4ccb-9659-140d0bf31b72-kube-api-access-pzmbq\") pod \"payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.705686 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.705633    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7616-f85a38"]
Sep 11 23:18:26.712483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.712043    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7616-f85a38"]
Sep 11 23:18:26.757017 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.756990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:26.772214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.772183    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:26.834538 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.834495    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7616-227f36"]
Sep 11 23:18:26.834879 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.834862    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4df7aba-7f26-418a-a15f-3109ca355767" containerName="test-curl-7616-f85a38"
Sep 11 23:18:26.834989 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.834883    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4df7aba-7f26-418a-a15f-3109ca355767" containerName="test-curl-7616-f85a38"
Sep 11 23:18:26.835047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.834990    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4df7aba-7f26-418a-a15f-3109ca355767" containerName="test-curl-7616-f85a38"
Sep 11 23:18:26.838030 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.838010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:26.846295 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.846269    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-227f36"]
Sep 11 23:18:26.857073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.857046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:26.994745 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:26.994713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djcv9\" (UniqueName: \"kubernetes.io/projected/a725a63d-6830-4273-a67c-fa9b81205cca-kube-api-access-djcv9\") pod \"test-curl-7616-227f36\" (UID: \"a725a63d-6830-4273-a67c-fa9b81205cca\") " pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:27.046182 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.046129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"]
Sep 11 23:18:27.048204 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:27.048177    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105e5c69_f920_4ccb_9659_140d0bf31b72.slice/crio-d33480463720e95dd626fe51dde36911746a84b4994f494741f077b6fe3390b3 WatchSource:0}: Error finding container d33480463720e95dd626fe51dde36911746a84b4994f494741f077b6fe3390b3: Status 404 returned error can't find the container with id d33480463720e95dd626fe51dde36911746a84b4994f494741f077b6fe3390b3
Sep 11 23:18:27.071785 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.071761    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:27.095856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.095795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djcv9\" (UniqueName: \"kubernetes.io/projected/a725a63d-6830-4273-a67c-fa9b81205cca-kube-api-access-djcv9\") pod \"test-curl-7616-227f36\" (UID: \"a725a63d-6830-4273-a67c-fa9b81205cca\") " pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:27.104211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.104187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8" event={"ID":"105e5c69-f920-4ccb-9659-140d0bf31b72","Type":"ContainerStarted","Data":"d33480463720e95dd626fe51dde36911746a84b4994f494741f077b6fe3390b3"}
Sep 11 23:18:27.104422 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:18:27.104404    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7616-f85a38" container="test-curl-7616-f85a38"
Sep 11 23:18:27.105062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.105029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djcv9\" (UniqueName: \"kubernetes.io/projected/a725a63d-6830-4273-a67c-fa9b81205cca-kube-api-access-djcv9\") pod \"test-curl-7616-227f36\" (UID: \"a725a63d-6830-4273-a67c-fa9b81205cca\") " pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:27.148046 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.148022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:27.305863 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.305826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:27.407558 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.407464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:27.417689 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.417661    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"]
Sep 11 23:18:27.421974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.421951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.424623 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.424604    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-92d8ce-dockercfg-v9v5r\""
Sep 11 23:18:27.425070 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.425054    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-92d8ce\""
Sep 11 23:18:27.442977 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.442948    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-227f36"]
Sep 11 23:18:27.445483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.445467    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"]
Sep 11 23:18:27.446987 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.446967    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"]
Sep 11 23:18:27.450333 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.450310    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.459464 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.459437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"]
Sep 11 23:18:27.499494 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.499466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f056b4ef-c290-4143-936b-d675cc2fd91b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.499615 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.499598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4dzk\" (UniqueName: \"kubernetes.io/projected/f056b4ef-c290-4143-936b-d675cc2fd91b-kube-api-access-c4dzk\") pod \"payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.601029 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.600994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4dzk\" (UniqueName: \"kubernetes.io/projected/f056b4ef-c290-4143-936b-d675cc2fd91b-kube-api-access-c4dzk\") pod \"payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.601183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.601043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c56da4-71e6-4785-a628-257c5b02f4b4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.601183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.601144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f056b4ef-c290-4143-936b-d675cc2fd91b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.601271 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.601214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcrvn\" (UniqueName: \"kubernetes.io/projected/e8c56da4-71e6-4785-a628-257c5b02f4b4-kube-api-access-qcrvn\") pod \"payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.601785 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.601767    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f056b4ef-c290-4143-936b-d675cc2fd91b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.613772 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.613738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4dzk\" (UniqueName: \"kubernetes.io/projected/f056b4ef-c290-4143-936b-d675cc2fd91b-kube-api-access-c4dzk\") pod \"payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.702567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.702482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c56da4-71e6-4785-a628-257c5b02f4b4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.702567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.702536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcrvn\" (UniqueName: \"kubernetes.io/projected/e8c56da4-71e6-4785-a628-257c5b02f4b4-kube-api-access-qcrvn\") pod \"payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.703184 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.703162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c56da4-71e6-4785-a628-257c5b02f4b4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.710829 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.710805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcrvn\" (UniqueName: \"kubernetes.io/projected/e8c56da4-71e6-4785-a628-257c5b02f4b4-kube-api-access-qcrvn\") pod \"payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.732333 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.732309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:27.762336 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.762312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:27.959881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:27.959786    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:28.007578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.007551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"]
Sep 11 23:18:28.009640 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:28.009616    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf056b4ef_c290_4143_936b_d675cc2fd91b.slice/crio-26aa5deee1b8decacc20b19727fec574aab1138c5daa0abbf1f9d80f9de918f1 WatchSource:0}: Error finding container 26aa5deee1b8decacc20b19727fec574aab1138c5daa0abbf1f9d80f9de918f1: Status 404 returned error can't find the container with id 26aa5deee1b8decacc20b19727fec574aab1138c5daa0abbf1f9d80f9de918f1
Sep 11 23:18:28.046042 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.046018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"]
Sep 11 23:18:28.047995 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:28.047965    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c56da4_71e6_4785_a628_257c5b02f4b4.slice/crio-432f6fbcf020e36a589d042b7b87e97622738a479a9fa9ce4f27a5daf03678c7 WatchSource:0}: Error finding container 432f6fbcf020e36a589d042b7b87e97622738a479a9fa9ce4f27a5daf03678c7: Status 404 returned error can't find the container with id 432f6fbcf020e36a589d042b7b87e97622738a479a9fa9ce4f27a5daf03678c7
Sep 11 23:18:28.111233 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.111199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-227f36" event={"ID":"a725a63d-6830-4273-a67c-fa9b81205cca","Type":"ContainerStarted","Data":"5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012"}
Sep 11 23:18:28.111371 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.111244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-227f36" event={"ID":"a725a63d-6830-4273-a67c-fa9b81205cca","Type":"ContainerStarted","Data":"3aa8a50030f9c9a67e10085b9cf58ee951c9ff9c5c898a06a3f937958e157956"}
Sep 11 23:18:28.113037 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.113011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb" event={"ID":"f056b4ef-c290-4143-936b-d675cc2fd91b","Type":"ContainerStarted","Data":"cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25"}
Sep 11 23:18:28.113150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.113060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb" event={"ID":"f056b4ef-c290-4143-936b-d675cc2fd91b","Type":"ContainerStarted","Data":"26aa5deee1b8decacc20b19727fec574aab1138c5daa0abbf1f9d80f9de918f1"}
Sep 11 23:18:28.113150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.113089    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:28.114753 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.114723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8" event={"ID":"105e5c69-f920-4ccb-9659-140d0bf31b72","Type":"ContainerStarted","Data":"ba4efe22f10fe4a744bb2e6c3f54b706cfa6c26d08a17ed7bdf8837cffe8ca69"}
Sep 11 23:18:28.115071 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.114784    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:28.116634 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.116606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8" event={"ID":"e8c56da4-71e6-4785-a628-257c5b02f4b4","Type":"ContainerStarted","Data":"5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5"}
Sep 11 23:18:28.116739 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.116656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8" event={"ID":"e8c56da4-71e6-4785-a628-257c5b02f4b4","Type":"ContainerStarted","Data":"432f6fbcf020e36a589d042b7b87e97622738a479a9fa9ce4f27a5daf03678c7"}
Sep 11 23:18:28.116739 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.116668    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7616-f85a38" podUID="e4df7aba-7f26-418a-a15f-3109ca355767" containerName="test-curl-7616-f85a38" containerID="cri-o://5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d" gracePeriod=2
Sep 11 23:18:28.116858 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.116765    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:28.129194 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.129152    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7616-227f36" podStartSLOduration=2.1291328099999998 podStartE2EDuration="2.12913281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:28.128183435 +0000 UTC m=+1023.340154604" watchObservedRunningTime="2026-09-11 23:18:28.12913281 +0000 UTC m=+1023.341103985"
Sep 11 23:18:28.148690 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.148642    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb" podStartSLOduration=1.148623639 podStartE2EDuration="1.148623639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:28.147742834 +0000 UTC m=+1023.359714009" watchObservedRunningTime="2026-09-11 23:18:28.148623639 +0000 UTC m=+1023.360594812"
Sep 11 23:18:28.165796 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.165747    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8" podStartSLOduration=2.16573277 podStartE2EDuration="2.16573277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:28.164499236 +0000 UTC m=+1023.376470410" watchObservedRunningTime="2026-09-11 23:18:28.16573277 +0000 UTC m=+1023.377703944"
Sep 11 23:18:28.181136 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.181093    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8" podStartSLOduration=1.1810776139999999 podStartE2EDuration="1.181077614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:28.180392707 +0000 UTC m=+1023.392363891" watchObservedRunningTime="2026-09-11 23:18:28.181077614 +0000 UTC m=+1023.393048788"
Sep 11 23:18:28.701720 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.701680    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7616-227f36"]
Sep 11 23:18:28.706188 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.706162    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7616-227f36"]
Sep 11 23:18:28.787657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.787619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:28.885765 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:28.885732    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:29.019857 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:29.019768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:29.121481 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:18:29.121451    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7616-227f36" container="test-curl-7616-227f36"
Sep 11 23:18:29.213964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:29.213903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:29.314422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:29.314386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:29.517411 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:29.517378    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:29.692338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:29.692243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:29.792922 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:29.792870    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:30.125853 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:30.125820    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7616-227f36" podUID="a725a63d-6830-4273-a67c-fa9b81205cca" containerName="test-curl-7616-227f36" containerID="cri-o://5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012" gracePeriod=2
Sep 11 23:18:30.279717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:30.279687    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:30.426357 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:30.426275    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm8wh\" (UniqueName: \"kubernetes.io/projected/e4df7aba-7f26-418a-a15f-3109ca355767-kube-api-access-hm8wh\") pod \"e4df7aba-7f26-418a-a15f-3109ca355767\" (UID: \"e4df7aba-7f26-418a-a15f-3109ca355767\") "
Sep 11 23:18:30.428300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:30.428278    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4df7aba-7f26-418a-a15f-3109ca355767-kube-api-access-hm8wh" (OuterVolumeSpecName: "kube-api-access-hm8wh") pod "e4df7aba-7f26-418a-a15f-3109ca355767" (UID: "e4df7aba-7f26-418a-a15f-3109ca355767"). InnerVolumeSpecName "kube-api-access-hm8wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:18:30.527541 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:30.527509    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm8wh\" (UniqueName: \"kubernetes.io/projected/e4df7aba-7f26-418a-a15f-3109ca355767-kube-api-access-hm8wh\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:18:31.131875 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.131835    2580 generic.go:356] "Generic (PLEG): container finished" podID="e4df7aba-7f26-418a-a15f-3109ca355767" containerID="5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d" exitCode=137
Sep 11 23:18:31.131875 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.131878    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:31.132356 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.131953    2580 scope.go:117] "RemoveContainer" containerID="5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d"
Sep 11 23:18:31.141332 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.141313    2580 scope.go:117] "RemoveContainer" containerID="5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d"
Sep 11 23:18:31.141601 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:18:31.141582    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d\": container with ID starting with 5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d not found: ID does not exist" containerID="5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d"
Sep 11 23:18:31.141653 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.141611    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d"} err="failed to get container status \"5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d\": rpc error: code = NotFound desc = could not find container \"5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d\": container with ID starting with 5c258d948c0783080e3c4de490f773c0c3e575619bfd29f4195345675239480d not found: ID does not exist"
Sep 11 23:18:31.143576 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:18:31.143553    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7616-f85a38\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="e4df7aba-7f26-418a-a15f-3109ca355767" pod="opendatahub/test-curl-7616-f85a38"
Sep 11 23:18:31.414729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.414641    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4df7aba-7f26-418a-a15f-3109ca355767" path="/var/lib/kubelet/pods/e4df7aba-7f26-418a-a15f-3109ca355767/volumes"
Sep 11 23:18:31.743684 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.743587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:31.858875 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.858834    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:31.975804 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:31.975767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:32.273003 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:32.272974    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:32.366497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:32.366460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:32.443944 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:32.443850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djcv9\" (UniqueName: \"kubernetes.io/projected/a725a63d-6830-4273-a67c-fa9b81205cca-kube-api-access-djcv9\") pod \"a725a63d-6830-4273-a67c-fa9b81205cca\" (UID: \"a725a63d-6830-4273-a67c-fa9b81205cca\") "
Sep 11 23:18:32.446119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:32.446079    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a725a63d-6830-4273-a67c-fa9b81205cca-kube-api-access-djcv9" (OuterVolumeSpecName: "kube-api-access-djcv9") pod "a725a63d-6830-4273-a67c-fa9b81205cca" (UID: "a725a63d-6830-4273-a67c-fa9b81205cca"). InnerVolumeSpecName "kube-api-access-djcv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:18:32.472713 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:32.472680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:32.544938 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:32.544893    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djcv9\" (UniqueName: \"kubernetes.io/projected/a725a63d-6830-4273-a67c-fa9b81205cca-kube-api-access-djcv9\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:18:33.145342 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:33.145304    2580 generic.go:356] "Generic (PLEG): container finished" podID="a725a63d-6830-4273-a67c-fa9b81205cca" containerID="5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012" exitCode=137
Sep 11 23:18:33.145529 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:33.145364    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:33.145529 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:33.145417    2580 scope.go:117] "RemoveContainer" containerID="5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012"
Sep 11 23:18:33.154082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:33.154069    2580 scope.go:117] "RemoveContainer" containerID="5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012"
Sep 11 23:18:33.154349 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:18:33.154330    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012\": container with ID starting with 5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012 not found: ID does not exist" containerID="5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012"
Sep 11 23:18:33.154409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:33.154358    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012"} err="failed to get container status \"5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012\": rpc error: code = NotFound desc = could not find container \"5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012\": container with ID starting with 5ec0ac17b048d656a9edcdf0007256909498ffcb8559d9f7328fcd631def5012 not found: ID does not exist"
Sep 11 23:18:33.157412 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:18:33.157389    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7616-227f36\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="a725a63d-6830-4273-a67c-fa9b81205cca" pod="opendatahub/test-curl-7616-227f36"
Sep 11 23:18:33.342430 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:33.342396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:33.413417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:33.413334    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a725a63d-6830-4273-a67c-fa9b81205cca" path="/var/lib/kubelet/pods/a725a63d-6830-4273-a67c-fa9b81205cca/volumes"
Sep 11 23:18:34.385267 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:34.385235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:34.478050 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:34.478025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:34.582412 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:34.582382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:34.931294 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:34.931264    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:35.028515 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.028484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:35.062246 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.062214    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"]
Sep 11 23:18:35.062537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.062525    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a725a63d-6830-4273-a67c-fa9b81205cca" containerName="test-curl-7616-227f36"
Sep 11 23:18:35.062587 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.062538    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a725a63d-6830-4273-a67c-fa9b81205cca" containerName="test-curl-7616-227f36"
Sep 11 23:18:35.062621 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.062603    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a725a63d-6830-4273-a67c-fa9b81205cca" containerName="test-curl-7616-227f36"
Sep 11 23:18:35.066995 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.066975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.069698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.069673    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0a7a66-dockercfg-299pc\""
Sep 11 23:18:35.069800 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.069673    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0a7a66\""
Sep 11 23:18:35.077015 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.076991    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"]
Sep 11 23:18:35.128604 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.128575    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"]
Sep 11 23:18:35.131626 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.131610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.142298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.142278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"]
Sep 11 23:18:35.165789 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.165767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-plugins-config-volume\") pod \"payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") " pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.165909 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.165803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gh6s\" (UniqueName: \"kubernetes.io/projected/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-kube-api-access-2gh6s\") pod \"payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") " pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.267037 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.266972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9smx\" (UniqueName: \"kubernetes.io/projected/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-kube-api-access-c9smx\") pod \"payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.267037 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.267009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.267221 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.267044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-plugins-config-volume\") pod \"payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") " pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.267221 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.267092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gh6s\" (UniqueName: \"kubernetes.io/projected/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-kube-api-access-2gh6s\") pod \"payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") " pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.267678 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.267658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-plugins-config-volume\") pod \"payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") " pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.275133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.275110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gh6s\" (UniqueName: \"kubernetes.io/projected/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-kube-api-access-2gh6s\") pod \"payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") " pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.367955 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.367906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9smx\" (UniqueName: \"kubernetes.io/projected/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-kube-api-access-c9smx\") pod \"payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.368078 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.367959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.368671 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.368645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.377099 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.377078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:35.377973 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.377953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9smx\" (UniqueName: \"kubernetes.io/projected/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-kube-api-access-c9smx\") pod \"payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.380733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.380702    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:35.441281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.441251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:35.651983 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.651886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"]
Sep 11 23:18:35.654269 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:35.654243    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8216dcf8_404f_4b5a_866e_8c2ff00a13a6.slice/crio-2ec31af005fb61b20540cdf763befe8304c756803260ba172a2f40f246cfce06 WatchSource:0}: Error finding container 2ec31af005fb61b20540cdf763befe8304c756803260ba172a2f40f246cfce06: Status 404 returned error can't find the container with id 2ec31af005fb61b20540cdf763befe8304c756803260ba172a2f40f246cfce06
Sep 11 23:18:35.714439 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:35.714364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"]
Sep 11 23:18:35.716368 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:35.716341    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8509cf72_d56b_4de1_ac7e_246e85f8b6d8.slice/crio-6ebe95b7983cdef7e4df87feec583d4b9c2890bc010532caf45a192a99b3e841 WatchSource:0}: Error finding container 6ebe95b7983cdef7e4df87feec583d4b9c2890bc010532caf45a192a99b3e841: Status 404 returned error can't find the container with id 6ebe95b7983cdef7e4df87feec583d4b9c2890bc010532caf45a192a99b3e841
Sep 11 23:18:36.161090 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.161051    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf" event={"ID":"8509cf72-d56b-4de1-ac7e-246e85f8b6d8","Type":"ContainerStarted","Data":"03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152"}
Sep 11 23:18:36.161261 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.161106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf" event={"ID":"8509cf72-d56b-4de1-ac7e-246e85f8b6d8","Type":"ContainerStarted","Data":"6ebe95b7983cdef7e4df87feec583d4b9c2890bc010532caf45a192a99b3e841"}
Sep 11 23:18:36.161261 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.161161    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:36.162844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.162811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm" event={"ID":"8216dcf8-404f-4b5a-866e-8c2ff00a13a6","Type":"ContainerStarted","Data":"7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95"}
Sep 11 23:18:36.162973 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.162859    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:36.162973 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.162881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm" event={"ID":"8216dcf8-404f-4b5a-866e-8c2ff00a13a6","Type":"ContainerStarted","Data":"2ec31af005fb61b20540cdf763befe8304c756803260ba172a2f40f246cfce06"}
Sep 11 23:18:36.178947 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.178063    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf" podStartSLOduration=1.178040937 podStartE2EDuration="1.178040937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:36.177634545 +0000 UTC m=+1031.389605719" watchObservedRunningTime="2026-09-11 23:18:36.178040937 +0000 UTC m=+1031.390012111"
Sep 11 23:18:36.196668 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.196624    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm" podStartSLOduration=1.1966112230000001 podStartE2EDuration="1.196611223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:36.195549662 +0000 UTC m=+1031.407520835" watchObservedRunningTime="2026-09-11 23:18:36.196611223 +0000 UTC m=+1031.408582396"
Sep 11 23:18:36.234051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.234023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:36.336083 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.336055    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:36.553443 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.553356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:36.832410 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.832383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:36.934335 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.934305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:36.945899 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.945870    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"]
Sep 11 23:18:36.949230 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.949214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:36.952208 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.952178    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a0c7930e\""
Sep 11 23:18:36.952338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.952264    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a0c7930e-dockercfg-dchwn\""
Sep 11 23:18:36.960475 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:36.960451    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"]
Sep 11 23:18:37.083185 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.083098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkkd7\" (UniqueName: \"kubernetes.io/projected/ba9c1950-6191-478f-9b6c-693947761857-kube-api-access-vkkd7\") pod \"payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:37.083185 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.083144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9c1950-6191-478f-9b6c-693947761857-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:37.184168 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.184133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkkd7\" (UniqueName: \"kubernetes.io/projected/ba9c1950-6191-478f-9b6c-693947761857-kube-api-access-vkkd7\") pod \"payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:37.184168 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.184174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9c1950-6191-478f-9b6c-693947761857-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:37.184831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.184812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9c1950-6191-478f-9b6c-693947761857-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:37.192239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.192219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkkd7\" (UniqueName: \"kubernetes.io/projected/ba9c1950-6191-478f-9b6c-693947761857-kube-api-access-vkkd7\") pod \"payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:37.260189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.260160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:37.530526 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.530477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"]
Sep 11 23:18:37.532999 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:37.532971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba9c1950_6191_478f_9b6c_693947761857.slice/crio-85657e6181964fb9b27f2396a8fbdea71e39200fd66ddabe7afa068209f2f936 WatchSource:0}: Error finding container 85657e6181964fb9b27f2396a8fbdea71e39200fd66ddabe7afa068209f2f936: Status 404 returned error can't find the container with id 85657e6181964fb9b27f2396a8fbdea71e39200fd66ddabe7afa068209f2f936
Sep 11 23:18:37.554012 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:37.553983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:38.064385 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.064349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:38.161410 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.161380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:38.173386 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.173357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2" event={"ID":"ba9c1950-6191-478f-9b6c-693947761857","Type":"ContainerStarted","Data":"c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325"}
Sep 11 23:18:38.173522 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.173412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2" event={"ID":"ba9c1950-6191-478f-9b6c-693947761857","Type":"ContainerStarted","Data":"85657e6181964fb9b27f2396a8fbdea71e39200fd66ddabe7afa068209f2f936"}
Sep 11 23:18:38.173522 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.173458    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:38.194120 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.194077    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2" podStartSLOduration=2.194063695 podStartE2EDuration="2.194063695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:38.192749138 +0000 UTC m=+1033.404720323" watchObservedRunningTime="2026-09-11 23:18:38.194063695 +0000 UTC m=+1033.406034917"
Sep 11 23:18:38.778925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.778888    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:38.914300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:38.914253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:39.123689 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.123661    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:18:39.123869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.123743    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:18:39.123869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.123785    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:18:39.377696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.377609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:39.478014 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.477981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:39.586114 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.586079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:39.766132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.766054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:39.874933 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.874888    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:39.968416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:39.968390    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:40.660419 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:40.660379    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:40.726270 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:40.726241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:40.919475 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:40.919392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:41.160016 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.159987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:41.278768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.278692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:41.327427 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.327398    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"]
Sep 11 23:18:41.331107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.331090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.340132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.340111    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-06ef79\""
Sep 11 23:18:41.347160 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.347137    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"]
Sep 11 23:18:41.424563 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.424522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gm8t\" (UniqueName: \"kubernetes.io/projected/34bf8857-aeb6-474d-9d58-52bf66df9534-kube-api-access-7gm8t\") pod \"payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.424735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.424613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bf8857-aeb6-474d-9d58-52bf66df9534-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.525834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.525798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bf8857-aeb6-474d-9d58-52bf66df9534-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.526022 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.525878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gm8t\" (UniqueName: \"kubernetes.io/projected/34bf8857-aeb6-474d-9d58-52bf66df9534-kube-api-access-7gm8t\") pod \"payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.526505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.526484    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bf8857-aeb6-474d-9d58-52bf66df9534-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.546320 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.546259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gm8t\" (UniqueName: \"kubernetes.io/projected/34bf8857-aeb6-474d-9d58-52bf66df9534-kube-api-access-7gm8t\") pod \"payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.640658 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.640630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:41.782169 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.782138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:41.912187 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:41.912158    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"]
Sep 11 23:18:41.916084 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:41.916054    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c WatchSource:0}: Error finding container 8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c: Status 404 returned error can't find the container with id 8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c
Sep 11 23:18:42.194622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:42.194537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252" event={"ID":"34bf8857-aeb6-474d-9d58-52bf66df9534","Type":"ContainerStarted","Data":"40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7"}
Sep 11 23:18:42.194622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:42.194584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252" event={"ID":"34bf8857-aeb6-474d-9d58-52bf66df9534","Type":"ContainerStarted","Data":"8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c"}
Sep 11 23:18:42.195038 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:42.194652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:42.214024 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:42.213980    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252" podStartSLOduration=1.213968729 podStartE2EDuration="1.213968729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:42.213370623 +0000 UTC m=+1037.425341798" watchObservedRunningTime="2026-09-11 23:18:42.213968729 +0000 UTC m=+1037.425939902"
Sep 11 23:18:42.586425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:42.586386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:42.728209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:42.728176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:43.575128 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:43.575094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:44.136210 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:44.136157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:44.287684 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:44.287653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:44.385968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:44.385907    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:44.483671 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:44.483598    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:44.771965 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:44.771860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:44.838989 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:44.838954    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:44.950162 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:44.950131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:45.567658 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:45.567625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:46.566467 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:46.566425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:47.170113 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:47.170081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:18:47.170336 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:47.170257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:18:47.479366 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:47.479281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:48.498238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:48.498209    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:48.589248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:48.589216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:49.182419 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:49.182388    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:18:49.772456 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:49.772416    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:50.785747 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:50.785712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:50.856144 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:50.856106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:51.460140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.460109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:51.556370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.556327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:51.576697 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.576666    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"]
Sep 11 23:18:51.580710 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.580685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.583669 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.583647    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-bfa694\""
Sep 11 23:18:51.591292 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.591268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"]
Sep 11 23:18:51.603138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.603114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468f8828-ee8f-4744-8ff8-b8385ac45ffb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.603238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.603217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxj6c\" (UniqueName: \"kubernetes.io/projected/468f8828-ee8f-4744-8ff8-b8385ac45ffb-kube-api-access-cxj6c\") pod \"payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.655067 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.655040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:51.704224 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.704203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468f8828-ee8f-4744-8ff8-b8385ac45ffb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.704327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.704268    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxj6c\" (UniqueName: \"kubernetes.io/projected/468f8828-ee8f-4744-8ff8-b8385ac45ffb-kube-api-access-cxj6c\") pod \"payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.704883 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.704866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468f8828-ee8f-4744-8ff8-b8385ac45ffb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.713727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.713667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxj6c\" (UniqueName: \"kubernetes.io/projected/468f8828-ee8f-4744-8ff8-b8385ac45ffb-kube-api-access-cxj6c\") pod \"payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.891806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.891763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:51.985999 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:51.985929    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:52.132687 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:52.132651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:52.166414 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:52.166389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"]
Sep 11 23:18:52.168412 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:18:52.168386    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468f8828_ee8f_4744_8ff8_b8385ac45ffb.slice/crio-91a5aed04f507a94bff3a42aef73c076d33ad9e1c684f22c7bf69bce25c58652 WatchSource:0}: Error finding container 91a5aed04f507a94bff3a42aef73c076d33ad9e1c684f22c7bf69bce25c58652: Status 404 returned error can't find the container with id 91a5aed04f507a94bff3a42aef73c076d33ad9e1c684f22c7bf69bce25c58652
Sep 11 23:18:52.242043 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:52.242020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t" event={"ID":"468f8828-ee8f-4744-8ff8-b8385ac45ffb","Type":"ContainerStarted","Data":"91a5aed04f507a94bff3a42aef73c076d33ad9e1c684f22c7bf69bce25c58652"}
Sep 11 23:18:53.025432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:53.025393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:53.201229 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:53.201201    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:18:53.248939 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:53.248872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t" event={"ID":"468f8828-ee8f-4744-8ff8-b8385ac45ffb","Type":"ContainerStarted","Data":"33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a"}
Sep 11 23:18:53.249077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:53.248953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:18:53.269333 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:53.269284    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t" podStartSLOduration=2.269269522 podStartE2EDuration="2.269269522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:53.268155532 +0000 UTC m=+1048.480126707" watchObservedRunningTime="2026-09-11 23:18:53.269269522 +0000 UTC m=+1048.481240687"
Sep 11 23:18:53.792377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:53.792339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:54.376807 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:54.376769    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:54.494728 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:54.494696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:54.650310 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:54.650221    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:54.750273 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:54.750219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:55.664617 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:55.664515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:56.589502 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:56.589437    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:57.361158 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:57.361109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:57.798089 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:57.798008    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:58.195669 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:58.195633    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:58.294827 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:58.294790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:58.398208 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:58.398174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:58.495612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:58.495523    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:18:59.402793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:18:59.402757    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:00.317612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:00.317570    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:01.083723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:01.083686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:01.199082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:01.199044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:01.294385 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:01.294352    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:01.442520 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:01.442440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:02.211940 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.211876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:02.261274 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.261235    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"]
Sep 11 23:19:02.261550 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.261520    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2" podUID="ba9c1950-6191-478f-9b6c-693947761857" containerName="payload-pre-processing" containerID="cri-o://c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325" gracePeriod=30
Sep 11 23:19:02.310674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.310642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:02.412398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.412262    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:02.515434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.515401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:02.516528 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.516510    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:19:02.601044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.601017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkkd7\" (UniqueName: \"kubernetes.io/projected/ba9c1950-6191-478f-9b6c-693947761857-kube-api-access-vkkd7\") pod \"ba9c1950-6191-478f-9b6c-693947761857\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") "
Sep 11 23:19:02.601211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.601158    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9c1950-6191-478f-9b6c-693947761857-plugins-config-volume\") pod \"ba9c1950-6191-478f-9b6c-693947761857\" (UID: \"ba9c1950-6191-478f-9b6c-693947761857\") "
Sep 11 23:19:02.601489 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.601457    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba9c1950-6191-478f-9b6c-693947761857-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba9c1950-6191-478f-9b6c-693947761857" (UID: "ba9c1950-6191-478f-9b6c-693947761857"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:19:02.603001 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.602974    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba9c1950-6191-478f-9b6c-693947761857-kube-api-access-vkkd7" (OuterVolumeSpecName: "kube-api-access-vkkd7") pod "ba9c1950-6191-478f-9b6c-693947761857" (UID: "ba9c1950-6191-478f-9b6c-693947761857"). InnerVolumeSpecName "kube-api-access-vkkd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:19:02.612339 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.612312    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:02.691363 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.691328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:02.702539 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.702507    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9c1950-6191-478f-9b6c-693947761857-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:19:02.702539 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.702532    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkkd7\" (UniqueName: \"kubernetes.io/projected/ba9c1950-6191-478f-9b6c-693947761857-kube-api-access-vkkd7\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:19:02.814137 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:02.814097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:03.179418 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.179328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:03.281484 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.281447    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:03.297960 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.297909    2580 generic.go:356] "Generic (PLEG): container finished" podID="ba9c1950-6191-478f-9b6c-693947761857" containerID="c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325" exitCode=0
Sep 11 23:19:03.298122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.297983    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"
Sep 11 23:19:03.298122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.298011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2" event={"ID":"ba9c1950-6191-478f-9b6c-693947761857","Type":"ContainerDied","Data":"c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325"}
Sep 11 23:19:03.298122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.298086    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2" event={"ID":"ba9c1950-6191-478f-9b6c-693947761857","Type":"ContainerDied","Data":"85657e6181964fb9b27f2396a8fbdea71e39200fd66ddabe7afa068209f2f936"}
Sep 11 23:19:03.298122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.298109    2580 scope.go:117] "RemoveContainer" containerID="c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325"
Sep 11 23:19:03.306867 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.306850    2580 scope.go:117] "RemoveContainer" containerID="c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325"
Sep 11 23:19:03.307154 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:19:03.307134    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325\": container with ID starting with c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325 not found: ID does not exist" containerID="c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325"
Sep 11 23:19:03.307207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.307164    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325"} err="failed to get container status \"c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325\": rpc error: code = NotFound desc = could not find container \"c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325\": container with ID starting with c4bbe65bb8d02994d00af3f8a9b9dbc397dcc0ca28b3c81ed1c07828abfe3325 not found: ID does not exist"
Sep 11 23:19:03.322085 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.322063    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"]
Sep 11 23:19:03.327250 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.327229    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a0c7930e-86d7fdb574-7pjk2"]
Sep 11 23:19:03.375679 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.375651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:03.413621 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.413594    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba9c1950-6191-478f-9b6c-693947761857" path="/var/lib/kubelet/pods/ba9c1950-6191-478f-9b6c-693947761857/volumes"
Sep 11 23:19:03.816538 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.816505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:03.957121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:03.957067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:04.059008 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:04.058971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:04.256169 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:04.256086    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:19:04.773815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:04.773777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:05.658249 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:05.658220    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:06.314677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:06.314651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:06.417400 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:06.417365    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:06.648455 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:06.648361    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:07.567545 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:07.567501    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:07.727785 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:07.727741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:07.827899 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:07.827806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:08.242886 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:08.242792    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:08.470383 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:08.470353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:09.382715 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:09.382680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:09.470300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:09.470267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:09.937714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:09.937679    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:10.087158 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.087125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:10.182989 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.182959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:10.203768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.203683    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"]
Sep 11 23:19:10.204056 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.204043    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba9c1950-6191-478f-9b6c-693947761857" containerName="payload-pre-processing"
Sep 11 23:19:10.204121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.204060    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba9c1950-6191-478f-9b6c-693947761857" containerName="payload-pre-processing"
Sep 11 23:19:10.204161 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.204125    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba9c1950-6191-478f-9b6c-693947761857" containerName="payload-pre-processing"
Sep 11 23:19:10.208409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.208393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.211554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.211526    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-7283a095\""
Sep 11 23:19:10.211685 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.211562    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-7283a095-dockercfg-4pgcl\""
Sep 11 23:19:10.219035 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.219015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"]
Sep 11 23:19:10.269007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.268981    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"]
Sep 11 23:19:10.272324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.272308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.274067 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.274036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h75nz\" (UniqueName: \"kubernetes.io/projected/e5e9220a-758c-4206-a434-2fcfedb9be63-kube-api-access-h75nz\") pod \"payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") " pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.274165 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.274140    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9220a-758c-4206-a434-2fcfedb9be63-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") " pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.283283 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.283263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"]
Sep 11 23:19:10.375198 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.375166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h75nz\" (UniqueName: \"kubernetes.io/projected/e5e9220a-758c-4206-a434-2fcfedb9be63-kube-api-access-h75nz\") pod \"payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") " pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.375353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.375231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl8l2\" (UniqueName: \"kubernetes.io/projected/59de47d6-0574-49b5-8e86-23e16c7ff9f8-kube-api-access-dl8l2\") pod \"payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.375353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.375264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9220a-758c-4206-a434-2fcfedb9be63-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") " pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.375353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.375284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59de47d6-0574-49b5-8e86-23e16c7ff9f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.375978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.375959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9220a-758c-4206-a434-2fcfedb9be63-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") " pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.384377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.384357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h75nz\" (UniqueName: \"kubernetes.io/projected/e5e9220a-758c-4206-a434-2fcfedb9be63-kube-api-access-h75nz\") pod \"payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") " pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.475806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.475729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl8l2\" (UniqueName: \"kubernetes.io/projected/59de47d6-0574-49b5-8e86-23e16c7ff9f8-kube-api-access-dl8l2\") pod \"payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.475806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.475781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59de47d6-0574-49b5-8e86-23e16c7ff9f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.476477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.476458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59de47d6-0574-49b5-8e86-23e16c7ff9f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.492590 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.492565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl8l2\" (UniqueName: \"kubernetes.io/projected/59de47d6-0574-49b5-8e86-23e16c7ff9f8-kube-api-access-dl8l2\") pod \"payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.518427 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.518403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:10.582767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.582672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:10.688134 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.688098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:10.820090 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.820055    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"]
Sep 11 23:19:10.824374 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:19:10.824350    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e9220a_758c_4206_a434_2fcfedb9be63.slice/crio-93299bff3b1db092e261a7c9c0afffcbb59feb11bcbf8659d79ed73d682cd09f WatchSource:0}: Error finding container 93299bff3b1db092e261a7c9c0afffcbb59feb11bcbf8659d79ed73d682cd09f: Status 404 returned error can't find the container with id 93299bff3b1db092e261a7c9c0afffcbb59feb11bcbf8659d79ed73d682cd09f
Sep 11 23:19:10.882189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:10.882167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"]
Sep 11 23:19:10.885931 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:19:10.885898    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59de47d6_0574_49b5_8e86_23e16c7ff9f8.slice/crio-dec48ee4dc97f0e9b4c565fc2dd3853577be80202a9824d8c5e1bc5cc1ddac0a WatchSource:0}: Error finding container dec48ee4dc97f0e9b4c565fc2dd3853577be80202a9824d8c5e1bc5cc1ddac0a: Status 404 returned error can't find the container with id dec48ee4dc97f0e9b4c565fc2dd3853577be80202a9824d8c5e1bc5cc1ddac0a
Sep 11 23:19:11.344922 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.344873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv" event={"ID":"e5e9220a-758c-4206-a434-2fcfedb9be63","Type":"ContainerStarted","Data":"b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df"}
Sep 11 23:19:11.345123 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.344949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv" event={"ID":"e5e9220a-758c-4206-a434-2fcfedb9be63","Type":"ContainerStarted","Data":"93299bff3b1db092e261a7c9c0afffcbb59feb11bcbf8659d79ed73d682cd09f"}
Sep 11 23:19:11.345123 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.344996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:11.346618 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.346593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2" event={"ID":"59de47d6-0574-49b5-8e86-23e16c7ff9f8","Type":"ContainerStarted","Data":"7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393"}
Sep 11 23:19:11.346732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.346638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2" event={"ID":"59de47d6-0574-49b5-8e86-23e16c7ff9f8","Type":"ContainerStarted","Data":"dec48ee4dc97f0e9b4c565fc2dd3853577be80202a9824d8c5e1bc5cc1ddac0a"}
Sep 11 23:19:11.346732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.346678    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:11.369568 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.369526    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv" podStartSLOduration=1.369513469 podStartE2EDuration="1.369513469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:19:11.367298483 +0000 UTC m=+1066.579269658" watchObservedRunningTime="2026-09-11 23:19:11.369513469 +0000 UTC m=+1066.581484642"
Sep 11 23:19:11.387246 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.387204    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2" podStartSLOduration=1.3871903589999999 podStartE2EDuration="1.387190359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:19:11.386212167 +0000 UTC m=+1066.598183341" watchObservedRunningTime="2026-09-11 23:19:11.387190359 +0000 UTC m=+1066.599161532"
Sep 11 23:19:11.549270 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:11.549232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:12.450888 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:12.450855    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:13.175882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:13.175846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:13.957564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:13.957524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:14.970619 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:14.970589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:22.353390 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:22.353313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:22.353753 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:22.353395    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:30.518711 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:30.518663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:31.806605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:31.806568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:32.721241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:32.721203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:33.445881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:33.445847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:34.200347 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:34.200311    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:34.948425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:34.948389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:35.536787 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:35.536754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:35.656050 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:35.656011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:36.523512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:36.523479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:37.463179 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:37.463143    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:38.265834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:38.265794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:38.956170 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:38.956131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:39.765336 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:39.765300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:40.605183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:40.605143    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:41.447949 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:41.447899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:42.166950 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:42.166897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:46.329702 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:46.329662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:46.387727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:46.387699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:47.452358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.452313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:47.475849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.475816    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"]
Sep 11 23:19:47.476109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.476083    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv" podUID="e5e9220a-758c-4206-a434-2fcfedb9be63" containerName="payload-processing" containerID="cri-o://b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df" gracePeriod=30
Sep 11 23:19:47.485523 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.485499    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"]
Sep 11 23:19:47.485731 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.485706    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2" podUID="59de47d6-0574-49b5-8e86-23e16c7ff9f8" containerName="payload-pre-processing" containerID="cri-o://7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393" gracePeriod=30
Sep 11 23:19:47.530228 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.530192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:47.633365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.633328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:47.694282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.694259    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:47.730719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.730695    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:47.767027 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.767000    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:47.787650 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.787620    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59de47d6-0574-49b5-8e86-23e16c7ff9f8-plugins-config-volume\") pod \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") "
Sep 11 23:19:47.787793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.787677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl8l2\" (UniqueName: \"kubernetes.io/projected/59de47d6-0574-49b5-8e86-23e16c7ff9f8-kube-api-access-dl8l2\") pod \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\" (UID: \"59de47d6-0574-49b5-8e86-23e16c7ff9f8\") "
Sep 11 23:19:47.787793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.787737    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h75nz\" (UniqueName: \"kubernetes.io/projected/e5e9220a-758c-4206-a434-2fcfedb9be63-kube-api-access-h75nz\") pod \"e5e9220a-758c-4206-a434-2fcfedb9be63\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") "
Sep 11 23:19:47.787950 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.787794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9220a-758c-4206-a434-2fcfedb9be63-plugins-config-volume\") pod \"e5e9220a-758c-4206-a434-2fcfedb9be63\" (UID: \"e5e9220a-758c-4206-a434-2fcfedb9be63\") "
Sep 11 23:19:47.788064 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.788038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59de47d6-0574-49b5-8e86-23e16c7ff9f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "59de47d6-0574-49b5-8e86-23e16c7ff9f8" (UID: "59de47d6-0574-49b5-8e86-23e16c7ff9f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:19:47.788641 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.788614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5e9220a-758c-4206-a434-2fcfedb9be63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5e9220a-758c-4206-a434-2fcfedb9be63" (UID: "e5e9220a-758c-4206-a434-2fcfedb9be63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:19:47.788940 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.788669    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59de47d6-0574-49b5-8e86-23e16c7ff9f8-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:19:47.792045 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.792016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59de47d6-0574-49b5-8e86-23e16c7ff9f8-kube-api-access-dl8l2" (OuterVolumeSpecName: "kube-api-access-dl8l2") pod "59de47d6-0574-49b5-8e86-23e16c7ff9f8" (UID: "59de47d6-0574-49b5-8e86-23e16c7ff9f8"). InnerVolumeSpecName "kube-api-access-dl8l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:19:47.792963 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.792570    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5e9220a-758c-4206-a434-2fcfedb9be63-kube-api-access-h75nz" (OuterVolumeSpecName: "kube-api-access-h75nz") pod "e5e9220a-758c-4206-a434-2fcfedb9be63" (UID: "e5e9220a-758c-4206-a434-2fcfedb9be63"). InnerVolumeSpecName "kube-api-access-h75nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:19:47.830011 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.829982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:47.889497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.889457    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl8l2\" (UniqueName: \"kubernetes.io/projected/59de47d6-0574-49b5-8e86-23e16c7ff9f8-kube-api-access-dl8l2\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:19:47.889497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.889489    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h75nz\" (UniqueName: \"kubernetes.io/projected/e5e9220a-758c-4206-a434-2fcfedb9be63-kube-api-access-h75nz\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:19:47.889497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:47.889499    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9220a-758c-4206-a434-2fcfedb9be63-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:19:48.127188 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.127150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:48.252373 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.252333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:48.514746 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.514711    2580 generic.go:356] "Generic (PLEG): container finished" podID="e5e9220a-758c-4206-a434-2fcfedb9be63" containerID="b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df" exitCode=0
Sep 11 23:19:48.515284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.514792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv" event={"ID":"e5e9220a-758c-4206-a434-2fcfedb9be63","Type":"ContainerDied","Data":"b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df"}
Sep 11 23:19:48.515434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.515336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv" event={"ID":"e5e9220a-758c-4206-a434-2fcfedb9be63","Type":"ContainerDied","Data":"93299bff3b1db092e261a7c9c0afffcbb59feb11bcbf8659d79ed73d682cd09f"}
Sep 11 23:19:48.515434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.514863    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"
Sep 11 23:19:48.515434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.515378    2580 scope.go:117] "RemoveContainer" containerID="b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df"
Sep 11 23:19:48.517473 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.517448    2580 generic.go:356] "Generic (PLEG): container finished" podID="59de47d6-0574-49b5-8e86-23e16c7ff9f8" containerID="7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393" exitCode=0
Sep 11 23:19:48.517605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.517493    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"
Sep 11 23:19:48.517605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.517579    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2" event={"ID":"59de47d6-0574-49b5-8e86-23e16c7ff9f8","Type":"ContainerDied","Data":"7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393"}
Sep 11 23:19:48.517735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.517620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2" event={"ID":"59de47d6-0574-49b5-8e86-23e16c7ff9f8","Type":"ContainerDied","Data":"dec48ee4dc97f0e9b4c565fc2dd3853577be80202a9824d8c5e1bc5cc1ddac0a"}
Sep 11 23:19:48.525150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.525129    2580 scope.go:117] "RemoveContainer" containerID="b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df"
Sep 11 23:19:48.525589 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:19:48.525441    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df\": container with ID starting with b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df not found: ID does not exist" containerID="b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df"
Sep 11 23:19:48.525589 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.525479    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df"} err="failed to get container status \"b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df\": rpc error: code = NotFound desc = could not find container \"b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df\": container with ID starting with b135e2367dd2b2b9ae3535371c6aedd4498bbc3d2792f88e5c8ed6c7361220df not found: ID does not exist"
Sep 11 23:19:48.525589 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.525506    2580 scope.go:117] "RemoveContainer" containerID="7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393"
Sep 11 23:19:48.535354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.535336    2580 scope.go:117] "RemoveContainer" containerID="7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393"
Sep 11 23:19:48.535623 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:19:48.535599    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393\": container with ID starting with 7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393 not found: ID does not exist" containerID="7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393"
Sep 11 23:19:48.535710 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.535625    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393"} err="failed to get container status \"7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393\": rpc error: code = NotFound desc = could not find container \"7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393\": container with ID starting with 7a8cfd0cd69f9e316467f6cbd394334faf9ef0cc492db8438ed6ffb06b870393 not found: ID does not exist"
Sep 11 23:19:48.551417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.551392    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"]
Sep 11 23:19:48.560604 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.560579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:48.560735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.560693    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7283a095-68d4db5698-cm9l2"]
Sep 11 23:19:48.570925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.570895    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"]
Sep 11 23:19:48.580302 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.580281    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7283a095-597bb8b67d-vwlxv"]
Sep 11 23:19:48.899975 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:48.899942    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:49.000968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:49.000936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:49.413735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:49.413700    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59de47d6-0574-49b5-8e86-23e16c7ff9f8" path="/var/lib/kubelet/pods/59de47d6-0574-49b5-8e86-23e16c7ff9f8/volumes"
Sep 11 23:19:49.414094 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:49.414080    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5e9220a-758c-4206-a434-2fcfedb9be63" path="/var/lib/kubelet/pods/e5e9220a-758c-4206-a434-2fcfedb9be63/volumes"
Sep 11 23:19:49.824945 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:49.824905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:50.764436 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:50.764400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:51.459244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:51.459202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:51.552269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:51.552233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:51.652899 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:51.652864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:52.459654 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.459623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:52.608023 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.607989    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"]
Sep 11 23:19:52.608525 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.608504    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e9220a-758c-4206-a434-2fcfedb9be63" containerName="payload-processing"
Sep 11 23:19:52.608606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.608530    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e9220a-758c-4206-a434-2fcfedb9be63" containerName="payload-processing"
Sep 11 23:19:52.608606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.608567    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59de47d6-0574-49b5-8e86-23e16c7ff9f8" containerName="payload-pre-processing"
Sep 11 23:19:52.608606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.608577    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="59de47d6-0574-49b5-8e86-23e16c7ff9f8" containerName="payload-pre-processing"
Sep 11 23:19:52.608707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.608697    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5e9220a-758c-4206-a434-2fcfedb9be63" containerName="payload-processing"
Sep 11 23:19:52.608740 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.608716    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="59de47d6-0574-49b5-8e86-23e16c7ff9f8" containerName="payload-pre-processing"
Sep 11 23:19:52.613456 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.613437    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.616349 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.616324    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-0a7a66\"/\"openshift-service-ca.crt\""
Sep 11 23:19:52.616349 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.616341    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-0a7a66\"/\"kube-root-ca.crt\""
Sep 11 23:19:52.617773 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.617740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-0a7a66\"/\"default-dockercfg-bqljd\""
Sep 11 23:19:52.617929 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.617837    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-0a7a66\"/\"autores-model-0a7a66-kserve-self-signed-certs\""
Sep 11 23:19:52.620543 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.620520    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"]
Sep 11 23:19:52.643834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.643809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:52.730554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.730468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-dshm\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.730554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.730509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qnqv\" (UniqueName: \"kubernetes.io/projected/945da5d8-0c33-4d03-9fbd-bd8706365ac4-kube-api-access-7qnqv\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.730554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.730542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-home\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.730765 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.730614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tls-certs\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.730765 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.730643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-model-cache\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.730765 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.730685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tmp-dir\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831310 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-dshm\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qnqv\" (UniqueName: \"kubernetes.io/projected/945da5d8-0c33-4d03-9fbd-bd8706365ac4-kube-api-access-7qnqv\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-home\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tls-certs\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-model-cache\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tmp-dir\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-model-cache\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-home\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.831972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.831896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tmp-dir\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.833654 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.833630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-dshm\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.833978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.833899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tls-certs\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.840073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.840053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qnqv\" (UniqueName: \"kubernetes.io/projected/945da5d8-0c33-4d03-9fbd-bd8706365ac4-kube-api-access-7qnqv\") pod \"autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") " pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:52.924853 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:52.924822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:19:53.195575 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.195545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"]
Sep 11 23:19:53.197971 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:19:53.197937    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod945da5d8_0c33_4d03_9fbd_bd8706365ac4.slice/crio-4e9dfd7564ca9d356b2f102bbc1fc74415ae0499dbb4cf9f8da532bc38ee32d3 WatchSource:0}: Error finding container 4e9dfd7564ca9d356b2f102bbc1fc74415ae0499dbb4cf9f8da532bc38ee32d3: Status 404 returned error can't find the container with id 4e9dfd7564ca9d356b2f102bbc1fc74415ae0499dbb4cf9f8da532bc38ee32d3
Sep 11 23:19:53.271614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.271582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:53.371155 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.371120    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:53.543108 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.543020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw" event={"ID":"945da5d8-0c33-4d03-9fbd-bd8706365ac4","Type":"ContainerStarted","Data":"1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c"}
Sep 11 23:19:53.543108 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.543075    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw" event={"ID":"945da5d8-0c33-4d03-9fbd-bd8706365ac4","Type":"ContainerStarted","Data":"4e9dfd7564ca9d356b2f102bbc1fc74415ae0499dbb4cf9f8da532bc38ee32d3"}
Sep 11 23:19:53.560730 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.560680    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw" podStartSLOduration=1.560664167 podStartE2EDuration="1.560664167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:19:53.559544726 +0000 UTC m=+1108.771515900" watchObservedRunningTime="2026-09-11 23:19:53.560664167 +0000 UTC m=+1108.772635340"
Sep 11 23:19:53.756906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.756876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:53.855123 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.855085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:53.898366 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.898333    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"]
Sep 11 23:19:53.901630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.901611    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:53.904979 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.904958    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c8789319-dockercfg-c4r25\""
Sep 11 23:19:53.905118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.905100    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c8789319\""
Sep 11 23:19:53.920011 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.919985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"]
Sep 11 23:19:53.941599 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.941574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/859c3e8b-a4be-4681-8cfc-7150618f7f8b-plugins-config-volume\") pod \"payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") " pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:53.941710 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:53.941610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgsph\" (UniqueName: \"kubernetes.io/projected/859c3e8b-a4be-4681-8cfc-7150618f7f8b-kube-api-access-kgsph\") pod \"payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") " pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:54.020353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.020315    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:54.042994 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.042964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/859c3e8b-a4be-4681-8cfc-7150618f7f8b-plugins-config-volume\") pod \"payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") " pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:54.043140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.043005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgsph\" (UniqueName: \"kubernetes.io/projected/859c3e8b-a4be-4681-8cfc-7150618f7f8b-kube-api-access-kgsph\") pod \"payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") " pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:54.043665 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.043647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/859c3e8b-a4be-4681-8cfc-7150618f7f8b-plugins-config-volume\") pod \"payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") " pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:54.051735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.051703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgsph\" (UniqueName: \"kubernetes.io/projected/859c3e8b-a4be-4681-8cfc-7150618f7f8b-kube-api-access-kgsph\") pod \"payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") " pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:54.182411 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.182321    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:54.211993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.211968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:54.504705 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.504679    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"]
Sep 11 23:19:54.506579 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:19:54.506548    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859c3e8b_a4be_4681_8cfc_7150618f7f8b.slice/crio-7e3c51c27f33269af004f10645a03a4b8e1ec4ba92f86b4013993a0739f8e29b WatchSource:0}: Error finding container 7e3c51c27f33269af004f10645a03a4b8e1ec4ba92f86b4013993a0739f8e29b: Status 404 returned error can't find the container with id 7e3c51c27f33269af004f10645a03a4b8e1ec4ba92f86b4013993a0739f8e29b
Sep 11 23:19:54.549782 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:54.549753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627" event={"ID":"859c3e8b-a4be-4681-8cfc-7150618f7f8b","Type":"ContainerStarted","Data":"7e3c51c27f33269af004f10645a03a4b8e1ec4ba92f86b4013993a0739f8e29b"}
Sep 11 23:19:55.159547 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:55.159498    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:55.555261 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:55.555166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627" event={"ID":"859c3e8b-a4be-4681-8cfc-7150618f7f8b","Type":"ContainerStarted","Data":"3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede"}
Sep 11 23:19:55.555629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:55.555272    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:19:55.578307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:55.578263    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627" podStartSLOduration=2.578247487 podStartE2EDuration="2.578247487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:19:55.577133945 +0000 UTC m=+1110.789105121" watchObservedRunningTime="2026-09-11 23:19:55.578247487 +0000 UTC m=+1110.790218662"
Sep 11 23:19:55.992274 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:55.992235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:19:56.694039 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:19:56.694009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:02.925603 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:02.925569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:20:02.926212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:02.925716    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:20:02.938015 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:02.937989    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:20:03.605930 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:03.605894    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:20:06.561632 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:06.561598    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:20:15.306864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:15.306823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:16.151340 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:16.151303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:16.944842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:16.944804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:17.843422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:17.843389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:18.672902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:18.672867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:19.385045 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:19.385011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:19.505905 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:19.505867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:19.607334 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:19.607299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:20.193826 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:20.193785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:20.994895 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:20.994862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:21.683650 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:21.683606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:22.551261 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:22.551225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:23.159946 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.159901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:23.189807 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.189778    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"]
Sep 11 23:20:23.190061 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.190006    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627" podUID="859c3e8b-a4be-4681-8cfc-7150618f7f8b" containerName="payload-processing" containerID="cri-o://3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede" gracePeriod=30
Sep 11 23:20:23.230399 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.230360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:23.330544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.330483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:23.431031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.430930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:23.452993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.452971    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:20:23.600112 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.600078    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/859c3e8b-a4be-4681-8cfc-7150618f7f8b-plugins-config-volume\") pod \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") "
Sep 11 23:20:23.600272 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.600139    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgsph\" (UniqueName: \"kubernetes.io/projected/859c3e8b-a4be-4681-8cfc-7150618f7f8b-kube-api-access-kgsph\") pod \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\" (UID: \"859c3e8b-a4be-4681-8cfc-7150618f7f8b\") "
Sep 11 23:20:23.600365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.600344    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/859c3e8b-a4be-4681-8cfc-7150618f7f8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "859c3e8b-a4be-4681-8cfc-7150618f7f8b" (UID: "859c3e8b-a4be-4681-8cfc-7150618f7f8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:23.600440 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.600423    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/859c3e8b-a4be-4681-8cfc-7150618f7f8b-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.602216 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.602190    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/859c3e8b-a4be-4681-8cfc-7150618f7f8b-kube-api-access-kgsph" (OuterVolumeSpecName: "kube-api-access-kgsph") pod "859c3e8b-a4be-4681-8cfc-7150618f7f8b" (UID: "859c3e8b-a4be-4681-8cfc-7150618f7f8b"). InnerVolumeSpecName "kube-api-access-kgsph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:23.694425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.694352    2580 generic.go:356] "Generic (PLEG): container finished" podID="859c3e8b-a4be-4681-8cfc-7150618f7f8b" containerID="3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede" exitCode=0
Sep 11 23:20:23.694425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.694409    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"
Sep 11 23:20:23.694596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.694432    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627" event={"ID":"859c3e8b-a4be-4681-8cfc-7150618f7f8b","Type":"ContainerDied","Data":"3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede"}
Sep 11 23:20:23.694596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.694495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627" event={"ID":"859c3e8b-a4be-4681-8cfc-7150618f7f8b","Type":"ContainerDied","Data":"7e3c51c27f33269af004f10645a03a4b8e1ec4ba92f86b4013993a0739f8e29b"}
Sep 11 23:20:23.694596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.694516    2580 scope.go:117] "RemoveContainer" containerID="3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede"
Sep 11 23:20:23.701007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.700985    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgsph\" (UniqueName: \"kubernetes.io/projected/859c3e8b-a4be-4681-8cfc-7150618f7f8b-kube-api-access-kgsph\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.703861 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.703841    2580 scope.go:117] "RemoveContainer" containerID="3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede"
Sep 11 23:20:23.704164 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:20:23.704144    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede\": container with ID starting with 3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede not found: ID does not exist" containerID="3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede"
Sep 11 23:20:23.704215 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.704173    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede"} err="failed to get container status \"3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede\": rpc error: code = NotFound desc = could not find container \"3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede\": container with ID starting with 3614802817a3f5383a2042182e4dccb9943ef83893b5965ad9f63fe540f27ede not found: ID does not exist"
Sep 11 23:20:23.720859 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.720836    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"]
Sep 11 23:20:23.724385 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.724364    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c8789319-6fb7cddbc5-zc627"]
Sep 11 23:20:23.875448 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:23.875419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:24.451311 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:24.451276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:24.584065 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:24.584010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:24.914761 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:24.914723    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:25.219854 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:25.219766    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:25.397578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:25.397535    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:25.415377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:25.415350    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="859c3e8b-a4be-4681-8cfc-7150618f7f8b" path="/var/lib/kubelet/pods/859c3e8b-a4be-4681-8cfc-7150618f7f8b/volumes"
Sep 11 23:20:25.569951 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:25.569921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:26.466576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:26.466533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:27.284288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:27.284246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:27.781462 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:27.781424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:27.876882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:27.876850    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:28.292041 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:28.292007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:29.146829 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:29.146792    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:29.877281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:29.877242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:30.686212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:30.686181    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:30.788036 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:30.788003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:31.277310 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.277270    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:31.372292 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.372255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:31.421320 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.421287    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"]
Sep 11 23:20:31.421644 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.421631    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="859c3e8b-a4be-4681-8cfc-7150618f7f8b" containerName="payload-processing"
Sep 11 23:20:31.421644 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.421645    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="859c3e8b-a4be-4681-8cfc-7150618f7f8b" containerName="payload-processing"
Sep 11 23:20:31.421733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.421703    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="859c3e8b-a4be-4681-8cfc-7150618f7f8b" containerName="payload-processing"
Sep 11 23:20:31.426178 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.426163    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.433625 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.433606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-7e6fa6af\""
Sep 11 23:20:31.433749 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.433704    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-7e6fa6af-dockercfg-2xgnz\""
Sep 11 23:20:31.440928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.440891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"]
Sep 11 23:20:31.459109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.459087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/636bc8d6-869c-4e96-8e77-5ffe8147b50f-plugins-config-volume\") pod \"payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") " pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.459196 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.459165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wsgl\" (UniqueName: \"kubernetes.io/projected/636bc8d6-869c-4e96-8e77-5ffe8147b50f-kube-api-access-5wsgl\") pod \"payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") " pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.559838 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.559754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/636bc8d6-869c-4e96-8e77-5ffe8147b50f-plugins-config-volume\") pod \"payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") " pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.559838 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.559814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wsgl\" (UniqueName: \"kubernetes.io/projected/636bc8d6-869c-4e96-8e77-5ffe8147b50f-kube-api-access-5wsgl\") pod \"payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") " pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.560545 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.560523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/636bc8d6-869c-4e96-8e77-5ffe8147b50f-plugins-config-volume\") pod \"payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") " pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.569097 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.569069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wsgl\" (UniqueName: \"kubernetes.io/projected/636bc8d6-869c-4e96-8e77-5ffe8147b50f-kube-api-access-5wsgl\") pod \"payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") " pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.606783 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.606754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:31.736281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.736257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:31.754691 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:31.754662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:32.014022 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:32.013999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"]
Sep 11 23:20:32.016182 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:20:32.016155    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636bc8d6_869c_4e96_8e77_5ffe8147b50f.slice/crio-7a4768353ee4df675daa0ffce8808b603614b2ffbfbeb8877acbee31d350936f WatchSource:0}: Error finding container 7a4768353ee4df675daa0ffce8808b603614b2ffbfbeb8877acbee31d350936f: Status 404 returned error can't find the container with id 7a4768353ee4df675daa0ffce8808b603614b2ffbfbeb8877acbee31d350936f
Sep 11 23:20:32.555974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:32.555942    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:32.736341 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:32.736308    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c" event={"ID":"636bc8d6-869c-4e96-8e77-5ffe8147b50f","Type":"ContainerStarted","Data":"0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e"}
Sep 11 23:20:32.736493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:32.736360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c" event={"ID":"636bc8d6-869c-4e96-8e77-5ffe8147b50f","Type":"ContainerStarted","Data":"7a4768353ee4df675daa0ffce8808b603614b2ffbfbeb8877acbee31d350936f"}
Sep 11 23:20:32.736493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:32.736390    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:32.757413 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:32.757366    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c" podStartSLOduration=1.7573523469999999 podStartE2EDuration="1.757352347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:20:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:20:32.756002117 +0000 UTC m=+1147.967973292" watchObservedRunningTime="2026-09-11 23:20:32.757352347 +0000 UTC m=+1147.969323546"
Sep 11 23:20:33.364223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:33.364186    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:35.246844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:35.246803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:35.862326 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:35.862291    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:35.885605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:35.885571    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"]
Sep 11 23:20:35.885824 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:35.885802    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm" podUID="8216dcf8-404f-4b5a-866e-8c2ff00a13a6" containerName="payload-processing" containerID="cri-o://7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95" gracePeriod=30
Sep 11 23:20:35.898321 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:35.898297    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"]
Sep 11 23:20:35.898623 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:35.898578    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf" podUID="8509cf72-d56b-4de1-ac7e-246e85f8b6d8" containerName="payload-pre-processing" containerID="cri-o://03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152" gracePeriod=30
Sep 11 23:20:35.963360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:35.963332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:36.065457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.065373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:36.143628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.143607    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:20:36.165695 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.165646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:36.167220 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.167196    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:20:36.197818 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.197788    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-plugins-config-volume\") pod \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") "
Sep 11 23:20:36.197974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.197825    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gh6s\" (UniqueName: \"kubernetes.io/projected/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-kube-api-access-2gh6s\") pod \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\" (UID: \"8216dcf8-404f-4b5a-866e-8c2ff00a13a6\") "
Sep 11 23:20:36.197974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.197870    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-plugins-config-volume\") pod \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") "
Sep 11 23:20:36.197974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.197895    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9smx\" (UniqueName: \"kubernetes.io/projected/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-kube-api-access-c9smx\") pod \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\" (UID: \"8509cf72-d56b-4de1-ac7e-246e85f8b6d8\") "
Sep 11 23:20:36.198254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.198224    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8216dcf8-404f-4b5a-866e-8c2ff00a13a6" (UID: "8216dcf8-404f-4b5a-866e-8c2ff00a13a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:36.198254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.198242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8509cf72-d56b-4de1-ac7e-246e85f8b6d8" (UID: "8509cf72-d56b-4de1-ac7e-246e85f8b6d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:36.199924 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.199886    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-kube-api-access-2gh6s" (OuterVolumeSpecName: "kube-api-access-2gh6s") pod "8216dcf8-404f-4b5a-866e-8c2ff00a13a6" (UID: "8216dcf8-404f-4b5a-866e-8c2ff00a13a6"). InnerVolumeSpecName "kube-api-access-2gh6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:36.200003 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.199958    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-kube-api-access-c9smx" (OuterVolumeSpecName: "kube-api-access-c9smx") pod "8509cf72-d56b-4de1-ac7e-246e85f8b6d8" (UID: "8509cf72-d56b-4de1-ac7e-246e85f8b6d8"). InnerVolumeSpecName "kube-api-access-c9smx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:36.298577 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.298555    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.298577 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.298575    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gh6s\" (UniqueName: \"kubernetes.io/projected/8216dcf8-404f-4b5a-866e-8c2ff00a13a6-kube-api-access-2gh6s\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.298894 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.298586    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.298894 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.298595    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9smx\" (UniqueName: \"kubernetes.io/projected/8509cf72-d56b-4de1-ac7e-246e85f8b6d8-kube-api-access-c9smx\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.432311 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.432249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:36.533155 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.533126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:36.755688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.755606    2580 generic.go:356] "Generic (PLEG): container finished" podID="8509cf72-d56b-4de1-ac7e-246e85f8b6d8" containerID="03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152" exitCode=0
Sep 11 23:20:36.755688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.755646    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"
Sep 11 23:20:36.755898 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.755684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf" event={"ID":"8509cf72-d56b-4de1-ac7e-246e85f8b6d8","Type":"ContainerDied","Data":"03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152"}
Sep 11 23:20:36.755898 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.755741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf" event={"ID":"8509cf72-d56b-4de1-ac7e-246e85f8b6d8","Type":"ContainerDied","Data":"6ebe95b7983cdef7e4df87feec583d4b9c2890bc010532caf45a192a99b3e841"}
Sep 11 23:20:36.755898 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.755765    2580 scope.go:117] "RemoveContainer" containerID="03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152"
Sep 11 23:20:36.757176 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.757152    2580 generic.go:356] "Generic (PLEG): container finished" podID="8216dcf8-404f-4b5a-866e-8c2ff00a13a6" containerID="7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95" exitCode=0
Sep 11 23:20:36.757280 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.757193    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"
Sep 11 23:20:36.757280 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.757267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm" event={"ID":"8216dcf8-404f-4b5a-866e-8c2ff00a13a6","Type":"ContainerDied","Data":"7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95"}
Sep 11 23:20:36.757376 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.757307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm" event={"ID":"8216dcf8-404f-4b5a-866e-8c2ff00a13a6","Type":"ContainerDied","Data":"2ec31af005fb61b20540cdf763befe8304c756803260ba172a2f40f246cfce06"}
Sep 11 23:20:36.765288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.765266    2580 scope.go:117] "RemoveContainer" containerID="03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152"
Sep 11 23:20:36.765628 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:20:36.765600    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152\": container with ID starting with 03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152 not found: ID does not exist" containerID="03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152"
Sep 11 23:20:36.765628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.765639    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152"} err="failed to get container status \"03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152\": rpc error: code = NotFound desc = could not find container \"03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152\": container with ID starting with 03f4b2c28fea60d249b32d37de49f753bcf1f1ef0289b36c4cbb102d3e97e152 not found: ID does not exist"
Sep 11 23:20:36.765842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.765658    2580 scope.go:117] "RemoveContainer" containerID="7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95"
Sep 11 23:20:36.773521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.773500    2580 scope.go:117] "RemoveContainer" containerID="7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95"
Sep 11 23:20:36.773746 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:20:36.773729    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95\": container with ID starting with 7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95 not found: ID does not exist" containerID="7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95"
Sep 11 23:20:36.773791 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.773752    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95"} err="failed to get container status \"7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95\": rpc error: code = NotFound desc = could not find container \"7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95\": container with ID starting with 7727e21b5ebd4aa3ea7ce997d03edf91b07aeb63f1525ebb17858d581a430a95 not found: ID does not exist"
Sep 11 23:20:36.793661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.793639    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"]
Sep 11 23:20:36.798590 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.798569    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a7a66-69d5685995-bg6rm"]
Sep 11 23:20:36.812821 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.812799    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"]
Sep 11 23:20:36.816313 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:36.816294    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a7a66-6f77b7d5b6-xbxqf"]
Sep 11 23:20:37.350515 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:37.350480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:37.414701 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:37.414672    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8216dcf8-404f-4b5a-866e-8c2ff00a13a6" path="/var/lib/kubelet/pods/8216dcf8-404f-4b5a-866e-8c2ff00a13a6/volumes"
Sep 11 23:20:37.415036 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:37.415023    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8509cf72-d56b-4de1-ac7e-246e85f8b6d8" path="/var/lib/kubelet/pods/8509cf72-d56b-4de1-ac7e-246e85f8b6d8/volumes"
Sep 11 23:20:37.450957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:37.450930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:38.067398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:38.067371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:38.833763 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:38.833719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:39.776792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:39.776751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:40.500399 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:40.500367    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:41.514974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:41.514935    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"]
Sep 11 23:20:41.613246 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:41.613212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:41.986433 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:41.986396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:42.832661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:42.832630    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:43.619575 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:43.619542    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:43.743609 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:43.743577    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:44.412427 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:44.412390    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:45.353719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:45.353686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:46.062856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.062823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:46.408821 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.408786    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"]
Sep 11 23:20:46.409132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.409120    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8509cf72-d56b-4de1-ac7e-246e85f8b6d8" containerName="payload-pre-processing"
Sep 11 23:20:46.409177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.409134    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8509cf72-d56b-4de1-ac7e-246e85f8b6d8" containerName="payload-pre-processing"
Sep 11 23:20:46.409177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.409153    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8216dcf8-404f-4b5a-866e-8c2ff00a13a6" containerName="payload-processing"
Sep 11 23:20:46.409177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.409159    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8216dcf8-404f-4b5a-866e-8c2ff00a13a6" containerName="payload-processing"
Sep 11 23:20:46.409273 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.409208    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8509cf72-d56b-4de1-ac7e-246e85f8b6d8" containerName="payload-pre-processing"
Sep 11 23:20:46.409273 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.409218    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8216dcf8-404f-4b5a-866e-8c2ff00a13a6" containerName="payload-processing"
Sep 11 23:20:46.413463 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.413445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.417707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.417681    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-92d8ce\"/\"default-dockercfg-rpfj8\""
Sep 11 23:20:46.417810 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.417736    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-92d8ce\"/\"test-model-92d8ce-kserve-self-signed-certs\""
Sep 11 23:20:46.417810 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.417741    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-92d8ce\"/\"openshift-service-ca.crt\""
Sep 11 23:20:46.417810 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.417783    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-92d8ce\"/\"kube-root-ca.crt\""
Sep 11 23:20:46.422762 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.422738    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"]
Sep 11 23:20:46.487180 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.487145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lf5b\" (UniqueName: \"kubernetes.io/projected/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-kube-api-access-9lf5b\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.487346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.487317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tmp-dir\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.487412 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.487387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-model-cache\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.487533 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.487434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-home\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.487533 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.487468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-dshm\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.487656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.487601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tls-certs\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.588512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.588483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tmp-dir\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.588717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.588522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-model-cache\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.588717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.588539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-home\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.588717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.588562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-dshm\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.588717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.588602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tls-certs\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.588717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.588635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lf5b\" (UniqueName: \"kubernetes.io/projected/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-kube-api-access-9lf5b\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.589008 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.588965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tmp-dir\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.589068 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.589020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-home\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.589114 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.589056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-model-cache\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.590818 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.590795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-dshm\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.591266 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.591249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tls-certs\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.596661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.596639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lf5b\" (UniqueName: \"kubernetes.io/projected/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-kube-api-access-9lf5b\") pod \"test-model-92d8ce-kserve-85fbcd9fcf-wst6w\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") " pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.707844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.707761    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:46.723682 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.723661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:46.930078 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.930040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:46.995407 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:46.995384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"]
Sep 11 23:20:46.997469 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:20:46.997437    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8145eb83_4d17_4fc1_94ab_8e8b6bf1630a.slice/crio-9490fc61f746dba4e1106e2bc1027863f34ab7fe231f4c3fe6ad8ab965727eb7 WatchSource:0}: Error finding container 9490fc61f746dba4e1106e2bc1027863f34ab7fe231f4c3fe6ad8ab965727eb7: Status 404 returned error can't find the container with id 9490fc61f746dba4e1106e2bc1027863f34ab7fe231f4c3fe6ad8ab965727eb7
Sep 11 23:20:47.811525 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:47.811446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w" event={"ID":"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a","Type":"ContainerStarted","Data":"c5b05445894d15b397113cd52abba6d80364aa84ec215711e857a190db49edf6"}
Sep 11 23:20:47.811525 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:47.811488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w" event={"ID":"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a","Type":"ContainerStarted","Data":"9490fc61f746dba4e1106e2bc1027863f34ab7fe231f4c3fe6ad8ab965727eb7"}
Sep 11 23:20:47.828634 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:47.828587    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w" podStartSLOduration=1.828572722 podStartE2EDuration="1.828572722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:20:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:20:47.828057459 +0000 UTC m=+1163.040028633" watchObservedRunningTime="2026-09-11 23:20:47.828572722 +0000 UTC m=+1163.040543896"
Sep 11 23:20:51.718326 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:51.718289    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:52.887815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:52.887783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:52.932316 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:52.932274    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"]
Sep 11 23:20:52.932512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:52.932484    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c" podUID="636bc8d6-869c-4e96-8e77-5ffe8147b50f" containerName="payload-processing" containerID="cri-o://0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e" gracePeriod=30
Sep 11 23:20:52.966494 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:52.966460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:53.026464 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.026424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:53.123198 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.123167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:53.183160 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.183136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:53.188622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.188591    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:53.244902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.244839    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/636bc8d6-869c-4e96-8e77-5ffe8147b50f-plugins-config-volume\") pod \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") "
Sep 11 23:20:53.244902 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.244889    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wsgl\" (UniqueName: \"kubernetes.io/projected/636bc8d6-869c-4e96-8e77-5ffe8147b50f-kube-api-access-5wsgl\") pod \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\" (UID: \"636bc8d6-869c-4e96-8e77-5ffe8147b50f\") "
Sep 11 23:20:53.245275 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.245246    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636bc8d6-869c-4e96-8e77-5ffe8147b50f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "636bc8d6-869c-4e96-8e77-5ffe8147b50f" (UID: "636bc8d6-869c-4e96-8e77-5ffe8147b50f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:53.248519 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.248488    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/636bc8d6-869c-4e96-8e77-5ffe8147b50f-kube-api-access-5wsgl" (OuterVolumeSpecName: "kube-api-access-5wsgl") pod "636bc8d6-869c-4e96-8e77-5ffe8147b50f" (UID: "636bc8d6-869c-4e96-8e77-5ffe8147b50f"). InnerVolumeSpecName "kube-api-access-5wsgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:53.274805 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.274774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:53.346121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.346092    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/636bc8d6-869c-4e96-8e77-5ffe8147b50f-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.346121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.346115    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wsgl\" (UniqueName: \"kubernetes.io/projected/636bc8d6-869c-4e96-8e77-5ffe8147b50f-kube-api-access-5wsgl\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.469271 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.469237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:53.841197 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.841159    2580 generic.go:356] "Generic (PLEG): container finished" podID="636bc8d6-869c-4e96-8e77-5ffe8147b50f" containerID="0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e" exitCode=0
Sep 11 23:20:53.841346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.841206    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"
Sep 11 23:20:53.841346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.841246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c" event={"ID":"636bc8d6-869c-4e96-8e77-5ffe8147b50f","Type":"ContainerDied","Data":"0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e"}
Sep 11 23:20:53.841346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.841301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c" event={"ID":"636bc8d6-869c-4e96-8e77-5ffe8147b50f","Type":"ContainerDied","Data":"7a4768353ee4df675daa0ffce8808b603614b2ffbfbeb8877acbee31d350936f"}
Sep 11 23:20:53.841346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.841330    2580 scope.go:117] "RemoveContainer" containerID="0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e"
Sep 11 23:20:53.850115 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.850095    2580 scope.go:117] "RemoveContainer" containerID="0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e"
Sep 11 23:20:53.850397 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:20:53.850378    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e\": container with ID starting with 0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e not found: ID does not exist" containerID="0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e"
Sep 11 23:20:53.850465 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.850412    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e"} err="failed to get container status \"0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e\": rpc error: code = NotFound desc = could not find container \"0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e\": container with ID starting with 0d81de222eb379370ed47aeb73accf8e16bd1fa54dd28b6932f9324fd12b384e not found: ID does not exist"
Sep 11 23:20:53.868827 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.868804    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"]
Sep 11 23:20:53.872878 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.872858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e6fa6af-dc5ff4fdf-gmb8c"]
Sep 11 23:20:53.927781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:53.927742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:54.021041 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:54.021005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:54.122160 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:54.122089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:54.571153 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:54.571126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:55.353567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:55.353538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:55.420268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:55.415408    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="636bc8d6-869c-4e96-8e77-5ffe8147b50f" path="/var/lib/kubelet/pods/636bc8d6-869c-4e96-8e77-5ffe8147b50f/volumes"
Sep 11 23:20:55.456220 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:55.456191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:56.155677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.155636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:56.534011 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.533906    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw" podUID="945da5d8-0c33-4d03-9fbd-bd8706365ac4" containerName="main" containerID="cri-o://1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c" gracePeriod=45
Sep 11 23:20:56.724309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.724279    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:56.724443 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.724355    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:56.736712 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.736687    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:56.781354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.781335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:20:56.844771 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.844741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:20:56.858053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.858025    2580 generic.go:356] "Generic (PLEG): container finished" podID="945da5d8-0c33-4d03-9fbd-bd8706365ac4" containerID="1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c" exitCode=0
Sep 11 23:20:56.858191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.858076    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"
Sep 11 23:20:56.858191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.858100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw" event={"ID":"945da5d8-0c33-4d03-9fbd-bd8706365ac4","Type":"ContainerDied","Data":"1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c"}
Sep 11 23:20:56.858191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.858140    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw" event={"ID":"945da5d8-0c33-4d03-9fbd-bd8706365ac4","Type":"ContainerDied","Data":"4e9dfd7564ca9d356b2f102bbc1fc74415ae0499dbb4cf9f8da532bc38ee32d3"}
Sep 11 23:20:56.858191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.858158    2580 scope.go:117] "RemoveContainer" containerID="1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c"
Sep 11 23:20:56.866562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.866545    2580 scope.go:117] "RemoveContainer" containerID="1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c"
Sep 11 23:20:56.866836 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:20:56.866820    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c\": container with ID starting with 1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c not found: ID does not exist" containerID="1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c"
Sep 11 23:20:56.867017 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.866843    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c"} err="failed to get container status \"1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c\": rpc error: code = NotFound desc = could not find container \"1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c\": container with ID starting with 1a5b183fa0c76d0f1112cd8d2eba428c509402cefe9919f4d0c1af01df76ad4c not found: ID does not exist"
Sep 11 23:20:56.870464 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.870440    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:20:56.873767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.873748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qnqv\" (UniqueName: \"kubernetes.io/projected/945da5d8-0c33-4d03-9fbd-bd8706365ac4-kube-api-access-7qnqv\") pod \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") "
Sep 11 23:20:56.873853 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.873801    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tmp-dir\") pod \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") "
Sep 11 23:20:56.873937 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.873862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-dshm\") pod \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") "
Sep 11 23:20:56.873937 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.873895    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-home\") pod \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") "
Sep 11 23:20:56.874052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.873964    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-model-cache\") pod \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") "
Sep 11 23:20:56.874052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.874008    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "945da5d8-0c33-4d03-9fbd-bd8706365ac4" (UID: "945da5d8-0c33-4d03-9fbd-bd8706365ac4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:20:56.874052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.874041    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tls-certs\") pod \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\" (UID: \"945da5d8-0c33-4d03-9fbd-bd8706365ac4\") "
Sep 11 23:20:56.874193 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.874159    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-home" (OuterVolumeSpecName: "home") pod "945da5d8-0c33-4d03-9fbd-bd8706365ac4" (UID: "945da5d8-0c33-4d03-9fbd-bd8706365ac4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:20:56.874193 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.874175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-model-cache" (OuterVolumeSpecName: "model-cache") pod "945da5d8-0c33-4d03-9fbd-bd8706365ac4" (UID: "945da5d8-0c33-4d03-9fbd-bd8706365ac4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:20:56.874430 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.874387    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:56.874516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.874437    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:56.874516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.874451    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:56.875956 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.875904    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945da5d8-0c33-4d03-9fbd-bd8706365ac4-kube-api-access-7qnqv" (OuterVolumeSpecName: "kube-api-access-7qnqv") pod "945da5d8-0c33-4d03-9fbd-bd8706365ac4" (UID: "945da5d8-0c33-4d03-9fbd-bd8706365ac4"). InnerVolumeSpecName "kube-api-access-7qnqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:56.876082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.876038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-dshm" (OuterVolumeSpecName: "dshm") pod "945da5d8-0c33-4d03-9fbd-bd8706365ac4" (UID: "945da5d8-0c33-4d03-9fbd-bd8706365ac4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:20:56.876082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.876051    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "945da5d8-0c33-4d03-9fbd-bd8706365ac4" (UID: "945da5d8-0c33-4d03-9fbd-bd8706365ac4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:20:56.975421 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.975384    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/945da5d8-0c33-4d03-9fbd-bd8706365ac4-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:56.975421 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.975410    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qnqv\" (UniqueName: \"kubernetes.io/projected/945da5d8-0c33-4d03-9fbd-bd8706365ac4-kube-api-access-7qnqv\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:56.975421 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:56.975420    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/945da5d8-0c33-4d03-9fbd-bd8706365ac4-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:20:57.182858 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:57.182832    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"]
Sep 11 23:20:57.189613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:57.189580    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-0a7a66/autores-model-0a7a66-kserve-7bbf6b5d59-w6xrw"]
Sep 11 23:20:57.413755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:57.413728    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945da5d8-0c33-4d03-9fbd-bd8706365ac4" path="/var/lib/kubelet/pods/945da5d8-0c33-4d03-9fbd-bd8706365ac4/volumes"
Sep 11 23:20:58.682678 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:20:58.682650    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:01.241020 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:01.240981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:01.347735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:01.347702    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:01.445836 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:01.445803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:01.520949 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:01.520848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:01.668733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:01.668701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:01.996625 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:01.996587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:13.307943 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:13.307892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:13.552091 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:13.552056    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:13.985312 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:13.985282    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:14.277923 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:14.277802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:14.536107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:14.536027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:17.045329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:17.045290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:17.765309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:17.765268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:17.860190 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:17.860154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:18.732080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:18.732047    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:19.664513 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:19.664482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:20.509053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:20.509011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:21.255121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.255087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:21.354370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.354335    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:21.419772 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.419730    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:21.752179 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.752147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:21.854015 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.853971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:21.858183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.858157    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"]
Sep 11 23:21:21.858484 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.858473    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945da5d8-0c33-4d03-9fbd-bd8706365ac4" containerName="main"
Sep 11 23:21:21.858530 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.858487    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="945da5d8-0c33-4d03-9fbd-bd8706365ac4" containerName="main"
Sep 11 23:21:21.858530 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.858497    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="636bc8d6-869c-4e96-8e77-5ffe8147b50f" containerName="payload-processing"
Sep 11 23:21:21.858530 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.858502    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="636bc8d6-869c-4e96-8e77-5ffe8147b50f" containerName="payload-processing"
Sep 11 23:21:21.858626 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.858566    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="945da5d8-0c33-4d03-9fbd-bd8706365ac4" containerName="main"
Sep 11 23:21:21.858626 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.858578    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="636bc8d6-869c-4e96-8e77-5ffe8147b50f" containerName="payload-processing"
Sep 11 23:21:21.862855 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.862839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:21.865526 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.865501    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-7719ed\""
Sep 11 23:21:21.873989 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.873965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"]
Sep 11 23:21:21.886713 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.886636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:21.886713 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.886683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8w2n\" (UniqueName: \"kubernetes.io/projected/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-kube-api-access-n8w2n\") pod \"payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:21.890171 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.890151    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"]
Sep 11 23:21:21.893387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.893372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:21.896139 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.896123    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-7719ed-dockercfg-wxzbm\""
Sep 11 23:21:21.902537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.902515    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"]
Sep 11 23:21:21.987242 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.987210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2krt5\" (UniqueName: \"kubernetes.io/projected/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-kube-api-access-2krt5\") pod \"payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") " pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:21.987384 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.987267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") " pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:21.987384 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.987336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:21.987505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.987388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2n\" (UniqueName: \"kubernetes.io/projected/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-kube-api-access-n8w2n\") pod \"payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:21.988023 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.987992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:21.995846 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:21.995827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8w2n\" (UniqueName: \"kubernetes.io/projected/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-kube-api-access-n8w2n\") pod \"payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:22.088182 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.088158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") " pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:22.088281 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.088219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2krt5\" (UniqueName: \"kubernetes.io/projected/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-kube-api-access-2krt5\") pod \"payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") " pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:22.088841 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.088822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") " pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:22.096835 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.096816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2krt5\" (UniqueName: \"kubernetes.io/projected/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-kube-api-access-2krt5\") pod \"payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") " pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:22.173103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.173080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:22.204448 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.204422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:22.445095 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.445072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"]
Sep 11 23:21:22.446549 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:21:22.446525    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6bfcd20_a3a9_452e_94fd_e7740a393ce3.slice/crio-4d7f7673e95a3ed7bb0de1baf584c01766fbeefd507ba78a060f340c6170d0a4 WatchSource:0}: Error finding container 4d7f7673e95a3ed7bb0de1baf584c01766fbeefd507ba78a060f340c6170d0a4: Status 404 returned error can't find the container with id 4d7f7673e95a3ed7bb0de1baf584c01766fbeefd507ba78a060f340c6170d0a4
Sep 11 23:21:22.487093 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.487075    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"]
Sep 11 23:21:22.488354 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:21:22.488332    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e6fb17_dc5f_4b73_b8de_94bbf5a0ce68.slice/crio-5f57a733ade97018ca30840d02e79851a39d462651e359cea8f79d4cee2f90ae WatchSource:0}: Error finding container 5f57a733ade97018ca30840d02e79851a39d462651e359cea8f79d4cee2f90ae: Status 404 returned error can't find the container with id 5f57a733ade97018ca30840d02e79851a39d462651e359cea8f79d4cee2f90ae
Sep 11 23:21:22.976455 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.976415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v" event={"ID":"f6bfcd20-a3a9-452e-94fd-e7740a393ce3","Type":"ContainerStarted","Data":"6ef73dd7764b1d992acf999e42e11166f793d24cfe27e2e2fa55de4774bd5dbc"}
Sep 11 23:21:22.976887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.976485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v" event={"ID":"f6bfcd20-a3a9-452e-94fd-e7740a393ce3","Type":"ContainerStarted","Data":"4d7f7673e95a3ed7bb0de1baf584c01766fbeefd507ba78a060f340c6170d0a4"}
Sep 11 23:21:22.976887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.976513    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:22.978146 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.978117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd" event={"ID":"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68","Type":"ContainerStarted","Data":"0dcd743d7667dd3e9e3a06d86c91c239faecdebc6481646b76760196bd0c3f00"}
Sep 11 23:21:22.978296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.978161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd" event={"ID":"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68","Type":"ContainerStarted","Data":"5f57a733ade97018ca30840d02e79851a39d462651e359cea8f79d4cee2f90ae"}
Sep 11 23:21:22.978296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.978245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:22.994348 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:22.994309    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v" podStartSLOduration=1.994295221 podStartE2EDuration="1.994295221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:21:22.993435297 +0000 UTC m=+1198.205406472" watchObservedRunningTime="2026-09-11 23:21:22.994295221 +0000 UTC m=+1198.206266427"
Sep 11 23:21:23.011997 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:23.011946    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd" podStartSLOduration=2.011897594 podStartE2EDuration="2.011897594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:21:23.011318457 +0000 UTC m=+1198.223289634" watchObservedRunningTime="2026-09-11 23:21:23.011897594 +0000 UTC m=+1198.223868771"
Sep 11 23:21:27.850402 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:27.850360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:33.713923 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:33.713884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:33.984887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:33.984809    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:21:33.984887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:33.984886    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:21:35.290015 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:35.289938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:35.548005 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:35.547925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:41.317749 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:41.317713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:43.344327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:43.344287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:44.168797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:44.168759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:44.855137 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:44.855104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:45.638724 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:45.638693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:46.342583 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:46.342551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:47.051101 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:47.051066    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:47.692364 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:47.692334    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:47.762612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:47.762577    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:49.738876 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:49.738835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:55.441573 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:55.441545    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:55.542680 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:55.542634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:21:55.644478 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:21:55.644411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:06.087065 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:06.087030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:06.806587 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:06.806556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:13.218789 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:13.218747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:13.327755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:13.327723    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:13.486592 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:13.486514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:13.593989 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:13.593955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:14.166897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:14.166856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:14.272353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:14.272318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:15.766373 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:15.766337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:21.756024 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:21.755938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:21.854310 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:21.854280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:22.398553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:22.398520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:28.444949 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:28.444893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:28.546907 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:28.546868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:28.667320 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:28.667288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:28.963718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:28.963680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:29.109425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:29.109392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:29.242733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:29.242644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:35.212361 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:35.212327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:35.329266 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:35.329228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:35.416876 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:35.416824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:36.258923 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:36.258865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:37.370844 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.370801    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:37.817001 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.816966    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"]
Sep 11 23:22:37.820596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.820572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:37.823780 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.823759    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-7719ed\"/\"openshift-service-ca.crt\""
Sep 11 23:22:37.825122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.825106    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-7719ed\"/\"kube-root-ca.crt\""
Sep 11 23:22:37.826576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.826552    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-7719ed\"/\"explicit-model-7719ed-kserve-self-signed-certs\""
Sep 11 23:22:37.826576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.826569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-7719ed\"/\"default-dockercfg-xdklq\""
Sep 11 23:22:37.832718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.832693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"]
Sep 11 23:22:37.932991 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.932955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tls-certs\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:37.932991 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.932989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-home\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:37.933177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.933007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmkdq\" (UniqueName: \"kubernetes.io/projected/5ac9a235-5bde-497f-90a7-6a7055e2e60b-kube-api-access-nmkdq\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:37.933177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.933110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-model-cache\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:37.933177 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.933143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tmp-dir\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:37.933277 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:37.933186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-dshm\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.034564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.034536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-dshm\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.034719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.034616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tls-certs\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.034719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.034644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-home\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.034719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.034666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmkdq\" (UniqueName: \"kubernetes.io/projected/5ac9a235-5bde-497f-90a7-6a7055e2e60b-kube-api-access-nmkdq\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.034845 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.034721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-model-cache\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.034845 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.034742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tmp-dir\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.035119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.035088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-home\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.035260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.035135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tmp-dir\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.035260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.035176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-model-cache\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.036924 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.036890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-dshm\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.037192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.037172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tls-certs\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.043477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.043455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmkdq\" (UniqueName: \"kubernetes.io/projected/5ac9a235-5bde-497f-90a7-6a7055e2e60b-kube-api-access-nmkdq\") pod \"explicit-model-7719ed-kserve-d4c77f79f-r44hc\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") " pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.123823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.123741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:38.129713 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.129694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:38.410793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.410705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:38.411621 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:38.411601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"]
Sep 11 23:22:38.414164 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:22:38.414135    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ac9a235_5bde_497f_90a7_6a7055e2e60b.slice/crio-31282a856553622bf4f997db92ff6dcbfa6a75f04232dfb280470eb6c4a2d3b5 WatchSource:0}: Error finding container 31282a856553622bf4f997db92ff6dcbfa6a75f04232dfb280470eb6c4a2d3b5: Status 404 returned error can't find the container with id 31282a856553622bf4f997db92ff6dcbfa6a75f04232dfb280470eb6c4a2d3b5
Sep 11 23:22:39.325958 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:39.325895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc" event={"ID":"5ac9a235-5bde-497f-90a7-6a7055e2e60b","Type":"ContainerStarted","Data":"fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6"}
Sep 11 23:22:39.325958 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:39.325964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc" event={"ID":"5ac9a235-5bde-497f-90a7-6a7055e2e60b","Type":"ContainerStarted","Data":"31282a856553622bf4f997db92ff6dcbfa6a75f04232dfb280470eb6c4a2d3b5"}
Sep 11 23:22:39.344714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:39.344664    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc" podStartSLOduration=2.344649126 podStartE2EDuration="2.344649126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:22:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:22:39.344586107 +0000 UTC m=+1274.556557262" watchObservedRunningTime="2026-09-11 23:22:39.344649126 +0000 UTC m=+1274.556620300"
Sep 11 23:22:39.484025 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:39.483983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:40.278587 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:40.278551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:40.377880 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:40.377849    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:40.633231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:40.633197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:40.903277 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:40.903198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:41.003469 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.003429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:22:41.020118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.020093    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"]
Sep 11 23:22:41.023668 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.023649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.027360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.027340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-1ebfb31f-dockercfg-qz5mv\""
Sep 11 23:22:41.027462 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.027381    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-1ebfb31f\""
Sep 11 23:22:41.040972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.040950    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"]
Sep 11 23:22:41.062224 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.062198    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"]
Sep 11 23:22:41.066296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.066278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.070899 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.070877    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"]
Sep 11 23:22:41.159320 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.159255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-plugins-config-volume\") pod \"payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") " pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.159320 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.159304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd5ac1d6-9458-43c9-808e-995e80c51423-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.159465 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.159348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66rwn\" (UniqueName: \"kubernetes.io/projected/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-kube-api-access-66rwn\") pod \"payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") " pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.159465 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.159371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96nc2\" (UniqueName: \"kubernetes.io/projected/dd5ac1d6-9458-43c9-808e-995e80c51423-kube-api-access-96nc2\") pod \"payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.259804 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.259777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-plugins-config-volume\") pod \"payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") " pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.259934 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.259838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd5ac1d6-9458-43c9-808e-995e80c51423-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.259934 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.259871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66rwn\" (UniqueName: \"kubernetes.io/projected/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-kube-api-access-66rwn\") pod \"payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") " pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.259934 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.259902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96nc2\" (UniqueName: \"kubernetes.io/projected/dd5ac1d6-9458-43c9-808e-995e80c51423-kube-api-access-96nc2\") pod \"payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.260452 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.260434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-plugins-config-volume\") pod \"payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") " pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.260639 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.260621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd5ac1d6-9458-43c9-808e-995e80c51423-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.268736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.268713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66rwn\" (UniqueName: \"kubernetes.io/projected/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-kube-api-access-66rwn\") pod \"payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") " pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.268848 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.268832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96nc2\" (UniqueName: \"kubernetes.io/projected/dd5ac1d6-9458-43c9-808e-995e80c51423-kube-api-access-96nc2\") pod \"payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.334244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.334221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:41.378849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.378622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:41.605486 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.605464    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"]
Sep 11 23:22:41.607543 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:22:41.607513    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5ac1d6_9458_43c9_808e_995e80c51423.slice/crio-b7d2ac997e81e030eb832964377733a0d2e6a0b1cdd6cdae4fa5e335e4338ba4 WatchSource:0}: Error finding container b7d2ac997e81e030eb832964377733a0d2e6a0b1cdd6cdae4fa5e335e4338ba4: Status 404 returned error can't find the container with id b7d2ac997e81e030eb832964377733a0d2e6a0b1cdd6cdae4fa5e335e4338ba4
Sep 11 23:22:41.669640 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:41.669611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"]
Sep 11 23:22:41.672287 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:22:41.672251    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bcdae1e_4f8d_4e78_8bea_322c327fe4fb.slice/crio-65085c24df6a1b6a929a1695090319834a8dab65a95ad4363bf7590475589520 WatchSource:0}: Error finding container 65085c24df6a1b6a929a1695090319834a8dab65a95ad4363bf7590475589520: Status 404 returned error can't find the container with id 65085c24df6a1b6a929a1695090319834a8dab65a95ad4363bf7590475589520
Sep 11 23:22:42.344692 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:42.344652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx" event={"ID":"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb","Type":"ContainerStarted","Data":"68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129"}
Sep 11 23:22:42.344887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:42.344711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx" event={"ID":"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb","Type":"ContainerStarted","Data":"65085c24df6a1b6a929a1695090319834a8dab65a95ad4363bf7590475589520"}
Sep 11 23:22:42.344887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:42.344839    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:22:42.346298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:42.346269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn" event={"ID":"dd5ac1d6-9458-43c9-808e-995e80c51423","Type":"ContainerStarted","Data":"a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233"}
Sep 11 23:22:42.346414 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:42.346314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn" event={"ID":"dd5ac1d6-9458-43c9-808e-995e80c51423","Type":"ContainerStarted","Data":"b7d2ac997e81e030eb832964377733a0d2e6a0b1cdd6cdae4fa5e335e4338ba4"}
Sep 11 23:22:42.346414 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:42.346337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:42.362801 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:42.362755    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx" podStartSLOduration=1.362742908 podStartE2EDuration="1.362742908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:22:42.36160186 +0000 UTC m=+1277.573573034" watchObservedRunningTime="2026-09-11 23:22:42.362742908 +0000 UTC m=+1277.574714113"
Sep 11 23:22:48.130125 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:48.130093    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:48.130686 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:48.130141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:48.143009 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:48.142984    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:48.161788 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:48.161736    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn" podStartSLOduration=7.161721121 podStartE2EDuration="7.161721121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:22:42.379743203 +0000 UTC m=+1277.591714378" watchObservedRunningTime="2026-09-11 23:22:48.161721121 +0000 UTC m=+1283.373692294"
Sep 11 23:22:48.384634 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:48.384547    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:22:53.352670 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:53.352637    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:22:53.353152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:22:53.352715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:23:00.010563 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:00.010514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:00.076786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:00.076742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:02.554367 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:02.554333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:03.387588 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:03.387550    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:04.249408 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:04.249376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:04.977868 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:04.977833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:05.273944 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:05.273836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:05.691344 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:05.691314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:05.872579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:05.872545    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:05.971612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:05.971514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:06.289768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:06.289681    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:06.740210 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:06.740173    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:07.443615 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:07.443583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:10.414516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.414482    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"]
Sep 11 23:23:10.418139 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.418116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.421019 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.420992    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"openshift-service-ca.crt\""
Sep 11 23:23:10.422552 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.422527    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"kube-root-ca.crt\""
Sep 11 23:23:10.422673 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.422638    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"default-dockercfg-hbcqb\""
Sep 11 23:23:10.422673 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.422645    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"e2e-conflict-a-8c150bb3-kserve-self-signed-certs\""
Sep 11 23:23:10.428614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.428583    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"]
Sep 11 23:23:10.442577 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.442545    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:10.511990 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.511958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-home\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.512138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.512008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-dshm\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.512138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.512039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tls-certs\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.512138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.512069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4wzf\" (UniqueName: \"kubernetes.io/projected/c119c1b3-bb24-4607-b461-ef4f7f348cbc-kube-api-access-t4wzf\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.512138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.512106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-model-cache\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.512138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.512125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tmp-dir\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613116 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tmp-dir\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-home\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-dshm\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tls-certs\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613413 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wzf\" (UniqueName: \"kubernetes.io/projected/c119c1b3-bb24-4607-b461-ef4f7f348cbc-kube-api-access-t4wzf\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613413 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-model-cache\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613526 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tmp-dir\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613848 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-home\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.613968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.613900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-model-cache\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.617960 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.616529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-dshm\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.618504 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.618479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tls-certs\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.622693 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.622668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4wzf\" (UniqueName: \"kubernetes.io/projected/c119c1b3-bb24-4607-b461-ef4f7f348cbc-kube-api-access-t4wzf\") pod \"e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.729805 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.729736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:10.872115 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:10.872079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:11.001323 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:11.001295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"]
Sep 11 23:23:11.002930 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:23:11.002888    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc119c1b3_bb24_4607_b461_ef4f7f348cbc.slice/crio-cf3847a54fc0e6739630ef9346d12c83fa10165b6f1ea6ea1aebe810221f8423 WatchSource:0}: Error finding container cf3847a54fc0e6739630ef9346d12c83fa10165b6f1ea6ea1aebe810221f8423: Status 404 returned error can't find the container with id cf3847a54fc0e6739630ef9346d12c83fa10165b6f1ea6ea1aebe810221f8423
Sep 11 23:23:11.482046 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:11.482004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l" event={"ID":"c119c1b3-bb24-4607-b461-ef4f7f348cbc","Type":"ContainerStarted","Data":"07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12"}
Sep 11 23:23:11.482428 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:11.482056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l" event={"ID":"c119c1b3-bb24-4607-b461-ef4f7f348cbc","Type":"ContainerStarted","Data":"cf3847a54fc0e6739630ef9346d12c83fa10165b6f1ea6ea1aebe810221f8423"}
Sep 11 23:23:11.501216 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:11.501165    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l" podStartSLOduration=1.501149237 podStartE2EDuration="1.501149237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:23:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:23:11.500071247 +0000 UTC m=+1306.712042421" watchObservedRunningTime="2026-09-11 23:23:11.501149237 +0000 UTC m=+1306.713120411"
Sep 11 23:23:11.753430 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:11.753354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:12.485303 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:12.485271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:12.794511 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:12.794426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:12.893629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:12.893588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:13.117155 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.117125    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"]
Sep 11 23:23:13.120564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.120547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"
Sep 11 23:23:13.123630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.123606    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vrx7m\""
Sep 11 23:23:13.123759 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.123646    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 23:23:13.129329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.129288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"]
Sep 11 23:23:13.137447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.137424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7fb448d-362b-459e-a3fd-55480bbbfbc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh\" (UID: \"c7fb448d-362b-459e-a3fd-55480bbbfbc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"
Sep 11 23:23:13.198543 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.198512    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"]
Sep 11 23:23:13.218618 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.218592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:13.238016 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.237991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7fb448d-362b-459e-a3fd-55480bbbfbc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh\" (UID: \"c7fb448d-362b-459e-a3fd-55480bbbfbc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"
Sep 11 23:23:13.238609 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.238586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7fb448d-362b-459e-a3fd-55480bbbfbc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh\" (UID: \"c7fb448d-362b-459e-a3fd-55480bbbfbc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"
Sep 11 23:23:13.426224 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.426132    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:13.433549 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.433524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"
Sep 11 23:23:13.712595 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.712564    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"]
Sep 11 23:23:13.714574 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:23:13.714546    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7fb448d_362b_459e_a3fd_55480bbbfbc6.slice/crio-df8c3e9d243fa1b4faf989133634066faa8fe17e927f444b7d582317e7e67658 WatchSource:0}: Error finding container df8c3e9d243fa1b4faf989133634066faa8fe17e927f444b7d582317e7e67658: Status 404 returned error can't find the container with id df8c3e9d243fa1b4faf989133634066faa8fe17e927f444b7d582317e7e67658
Sep 11 23:23:13.716648 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:13.716631    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:23:14.384495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:14.384453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:14.501839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:14.501808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh" event={"ID":"c7fb448d-362b-459e-a3fd-55480bbbfbc6","Type":"ContainerStarted","Data":"df8c3e9d243fa1b4faf989133634066faa8fe17e927f444b7d582317e7e67658"}
Sep 11 23:23:15.179975 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:15.179939    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:15.886308 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:15.886256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:16.513568 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:16.513532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh" event={"ID":"c7fb448d-362b-459e-a3fd-55480bbbfbc6","Type":"ContainerStarted","Data":"745d75f3a80d7b7a00f0ac7285959aade2f3f65b99a31bfa37e5162c23d1626d"}
Sep 11 23:23:16.531780 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:16.531729    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh" podStartSLOduration=1.111012218 podStartE2EDuration="3.531713099s" totalImagesPullingTime="2.420700881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:23:16.530630477 +0000 UTC m=+1311.742601653" watchObservedRunningTime="2026-09-11 23:23:16.531713099 +0000 UTC m=+1311.743684345"
Sep 11 23:23:16.562607 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:16.562578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:17.307973 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:17.307940    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:17.519565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:17.519533    2580 generic.go:356] "Generic (PLEG): container finished" podID="c7fb448d-362b-459e-a3fd-55480bbbfbc6" containerID="745d75f3a80d7b7a00f0ac7285959aade2f3f65b99a31bfa37e5162c23d1626d" exitCode=0
Sep 11 23:23:17.519961 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:17.519573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh" event={"ID":"c7fb448d-362b-459e-a3fd-55480bbbfbc6","Type":"ContainerDied","Data":"745d75f3a80d7b7a00f0ac7285959aade2f3f65b99a31bfa37e5162c23d1626d"}
Sep 11 23:23:18.155469 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:18.155426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:18.667410 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:18.667388    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"
Sep 11 23:23:18.789967 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:18.789894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7fb448d-362b-459e-a3fd-55480bbbfbc6-maas-api-service-ca\") pod \"c7fb448d-362b-459e-a3fd-55480bbbfbc6\" (UID: \"c7fb448d-362b-459e-a3fd-55480bbbfbc6\") "
Sep 11 23:23:18.790227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:18.790205    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7fb448d-362b-459e-a3fd-55480bbbfbc6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7fb448d-362b-459e-a3fd-55480bbbfbc6" (UID: "c7fb448d-362b-459e-a3fd-55480bbbfbc6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:23:18.890882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:18.890844    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7fb448d-362b-459e-a3fd-55480bbbfbc6-maas-api-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:18.958773 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:18.958740    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:19.530341 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:19.530308    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh" event={"ID":"c7fb448d-362b-459e-a3fd-55480bbbfbc6","Type":"ContainerDied","Data":"df8c3e9d243fa1b4faf989133634066faa8fe17e927f444b7d582317e7e67658"}
Sep 11 23:23:19.530341 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:19.530336    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"
Sep 11 23:23:19.530588 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:19.530355    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df8c3e9d243fa1b4faf989133634066faa8fe17e927f444b7d582317e7e67658"
Sep 11 23:23:20.682572 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:20.682539    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:20.730545 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:20.730512    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:20.730676 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:20.730564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:20.742931 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:20.742891    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:21.378605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.378569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:21.416965 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.416931    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"]
Sep 11 23:23:21.417490 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.417173    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd" podUID="f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" containerName="payload-processing" containerID="cri-o://0dcd743d7667dd3e9e3a06d86c91c239faecdebc6481646b76760196bd0c3f00" gracePeriod=30
Sep 11 23:23:21.423136 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.423109    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"]
Sep 11 23:23:21.423341 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.423320    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v" podUID="f6bfcd20-a3a9-452e-94fd-e7740a393ce3" containerName="payload-pre-processing" containerID="cri-o://6ef73dd7764b1d992acf999e42e11166f793d24cfe27e2e2fa55de4774bd5dbc" gracePeriod=30
Sep 11 23:23:21.462334 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.462304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:21.542393 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.542367    2580 generic.go:356] "Generic (PLEG): container finished" podID="f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" containerID="0dcd743d7667dd3e9e3a06d86c91c239faecdebc6481646b76760196bd0c3f00" exitCode=0
Sep 11 23:23:21.542504 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.542449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd" event={"ID":"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68","Type":"ContainerDied","Data":"0dcd743d7667dd3e9e3a06d86c91c239faecdebc6481646b76760196bd0c3f00"}
Sep 11 23:23:21.544370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.544347    2580 generic.go:356] "Generic (PLEG): container finished" podID="f6bfcd20-a3a9-452e-94fd-e7740a393ce3" containerID="6ef73dd7764b1d992acf999e42e11166f793d24cfe27e2e2fa55de4774bd5dbc" exitCode=0
Sep 11 23:23:21.544488 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.544430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v" event={"ID":"f6bfcd20-a3a9-452e-94fd-e7740a393ce3","Type":"ContainerDied","Data":"6ef73dd7764b1d992acf999e42e11166f793d24cfe27e2e2fa55de4774bd5dbc"}
Sep 11 23:23:21.559236 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.559190    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:23:21.588622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.588547    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:21.683235 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.683202    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:23:21.684924 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.684881    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:21.718197 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.718172    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:23:21.718290 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.718220    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2krt5\" (UniqueName: \"kubernetes.io/projected/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-kube-api-access-2krt5\") pod \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") "
Sep 11 23:23:21.718340 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.718322    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-plugins-config-volume\") pod \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\" (UID: \"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68\") "
Sep 11 23:23:21.718761 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.718729    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" (UID: "f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:23:21.720188 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.720166    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-kube-api-access-2krt5" (OuterVolumeSpecName: "kube-api-access-2krt5") pod "f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" (UID: "f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68"). InnerVolumeSpecName "kube-api-access-2krt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:23:21.819684 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.819653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8w2n\" (UniqueName: \"kubernetes.io/projected/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-kube-api-access-n8w2n\") pod \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") "
Sep 11 23:23:21.819828 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.819724    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-plugins-config-volume\") pod \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\" (UID: \"f6bfcd20-a3a9-452e-94fd-e7740a393ce3\") "
Sep 11 23:23:21.819900 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.819889    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:21.819964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.819899    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2krt5\" (UniqueName: \"kubernetes.io/projected/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68-kube-api-access-2krt5\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:21.820089 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.820066    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6bfcd20-a3a9-452e-94fd-e7740a393ce3" (UID: "f6bfcd20-a3a9-452e-94fd-e7740a393ce3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:23:21.821698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.821677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-kube-api-access-n8w2n" (OuterVolumeSpecName: "kube-api-access-n8w2n") pod "f6bfcd20-a3a9-452e-94fd-e7740a393ce3" (UID: "f6bfcd20-a3a9-452e-94fd-e7740a393ce3"). InnerVolumeSpecName "kube-api-access-n8w2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:23:21.921121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.921051    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8w2n\" (UniqueName: \"kubernetes.io/projected/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-kube-api-access-n8w2n\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:21.921121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.921078    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bfcd20-a3a9-452e-94fd-e7740a393ce3-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:21.967094 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:21.967068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:22.128461 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.128407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:22.550385 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.550360    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"
Sep 11 23:23:22.550563 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.550401    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v" event={"ID":"f6bfcd20-a3a9-452e-94fd-e7740a393ce3","Type":"ContainerDied","Data":"4d7f7673e95a3ed7bb0de1baf584c01766fbeefd507ba78a060f340c6170d0a4"}
Sep 11 23:23:22.550563 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.550456    2580 scope.go:117] "RemoveContainer" containerID="6ef73dd7764b1d992acf999e42e11166f793d24cfe27e2e2fa55de4774bd5dbc"
Sep 11 23:23:22.551863 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.551849    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"
Sep 11 23:23:22.551992 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.551889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd" event={"ID":"f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68","Type":"ContainerDied","Data":"5f57a733ade97018ca30840d02e79851a39d462651e359cea8f79d4cee2f90ae"}
Sep 11 23:23:22.562694 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.562675    2580 scope.go:117] "RemoveContainer" containerID="0dcd743d7667dd3e9e3a06d86c91c239faecdebc6481646b76760196bd0c3f00"
Sep 11 23:23:22.580850 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.580828    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"]
Sep 11 23:23:22.589795 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.589769    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7719ed-56dcdfbd4b-7929v"]
Sep 11 23:23:22.602466 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.602443    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"]
Sep 11 23:23:22.606773 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.606753    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7719ed-65dd8b78bf-8nrmd"]
Sep 11 23:23:22.857316 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:22.857272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:23.414276 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:23.414244    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" path="/var/lib/kubelet/pods/f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68/volumes"
Sep 11 23:23:23.414607 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:23.414595    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bfcd20-a3a9-452e-94fd-e7740a393ce3" path="/var/lib/kubelet/pods/f6bfcd20-a3a9-452e-94fd-e7740a393ce3/volumes"
Sep 11 23:23:23.600258 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:23.600228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:24.460662 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:24.460618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:25.082820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.082788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:25.106439 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.106410    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"]
Sep 11 23:23:25.106695 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.106670    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx" podUID="6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" containerName="payload-processing" containerID="cri-o://68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129" gracePeriod=30
Sep 11 23:23:25.115185 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.115160    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"]
Sep 11 23:23:25.115384 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.115356    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn" podUID="dd5ac1d6-9458-43c9-808e-995e80c51423" containerName="payload-pre-processing" containerID="cri-o://a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233" gracePeriod=30
Sep 11 23:23:25.148821 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.148795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:25.209549 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.209525    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:25.291314 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.291266    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"]
Sep 11 23:23:25.294628 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.293996    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ebfb31f-86xkh"]
Sep 11 23:23:25.294948 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.294868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:25.354955 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.354920    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:25.356685 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.356667    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:23:25.391239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.391218    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:23:25.416286 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.416256    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7fb448d-362b-459e-a3fd-55480bbbfbc6" path="/var/lib/kubelet/pods/c7fb448d-362b-459e-a3fd-55480bbbfbc6/volumes"
Sep 11 23:23:25.419698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.419662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:25.455906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.455881    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66rwn\" (UniqueName: \"kubernetes.io/projected/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-kube-api-access-66rwn\") pod \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") "
Sep 11 23:23:25.456064 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.455951    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-plugins-config-volume\") pod \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\" (UID: \"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb\") "
Sep 11 23:23:25.456064 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.455975    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd5ac1d6-9458-43c9-808e-995e80c51423-plugins-config-volume\") pod \"dd5ac1d6-9458-43c9-808e-995e80c51423\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") "
Sep 11 23:23:25.456187 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.456132    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96nc2\" (UniqueName: \"kubernetes.io/projected/dd5ac1d6-9458-43c9-808e-995e80c51423-kube-api-access-96nc2\") pod \"dd5ac1d6-9458-43c9-808e-995e80c51423\" (UID: \"dd5ac1d6-9458-43c9-808e-995e80c51423\") "
Sep 11 23:23:25.456306 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.456285    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd5ac1d6-9458-43c9-808e-995e80c51423-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd5ac1d6-9458-43c9-808e-995e80c51423" (UID: "dd5ac1d6-9458-43c9-808e-995e80c51423"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:23:25.456306 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.456292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" (UID: "6bcdae1e-4f8d-4e78-8bea-322c327fe4fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:23:25.456508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.456491    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.456553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.456511    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd5ac1d6-9458-43c9-808e-995e80c51423-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.457939 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.457904    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-kube-api-access-66rwn" (OuterVolumeSpecName: "kube-api-access-66rwn") pod "6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" (UID: "6bcdae1e-4f8d-4e78-8bea-322c327fe4fb"). InnerVolumeSpecName "kube-api-access-66rwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:23:25.458081 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.458061    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd5ac1d6-9458-43c9-808e-995e80c51423-kube-api-access-96nc2" (OuterVolumeSpecName: "kube-api-access-96nc2") pod "dd5ac1d6-9458-43c9-808e-995e80c51423" (UID: "dd5ac1d6-9458-43c9-808e-995e80c51423"). InnerVolumeSpecName "kube-api-access-96nc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:23:25.557894 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.557863    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66rwn\" (UniqueName: \"kubernetes.io/projected/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb-kube-api-access-66rwn\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.558208 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.557896    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96nc2\" (UniqueName: \"kubernetes.io/projected/dd5ac1d6-9458-43c9-808e-995e80c51423-kube-api-access-96nc2\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.569683 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.569653    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" containerID="68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129" exitCode=0
Sep 11 23:23:25.569800 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.569702    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"
Sep 11 23:23:25.569800 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.569737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx" event={"ID":"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb","Type":"ContainerDied","Data":"68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129"}
Sep 11 23:23:25.569800 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.569789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx" event={"ID":"6bcdae1e-4f8d-4e78-8bea-322c327fe4fb","Type":"ContainerDied","Data":"65085c24df6a1b6a929a1695090319834a8dab65a95ad4363bf7590475589520"}
Sep 11 23:23:25.569961 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.569812    2580 scope.go:117] "RemoveContainer" containerID="68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129"
Sep 11 23:23:25.571213 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.571193    2580 generic.go:356] "Generic (PLEG): container finished" podID="dd5ac1d6-9458-43c9-808e-995e80c51423" containerID="a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233" exitCode=0
Sep 11 23:23:25.571324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.571241    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"
Sep 11 23:23:25.571324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.571249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn" event={"ID":"dd5ac1d6-9458-43c9-808e-995e80c51423","Type":"ContainerDied","Data":"a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233"}
Sep 11 23:23:25.571324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.571290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn" event={"ID":"dd5ac1d6-9458-43c9-808e-995e80c51423","Type":"ContainerDied","Data":"b7d2ac997e81e030eb832964377733a0d2e6a0b1cdd6cdae4fa5e335e4338ba4"}
Sep 11 23:23:25.577842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.577825    2580 scope.go:117] "RemoveContainer" containerID="68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129"
Sep 11 23:23:25.578105 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:23:25.578089    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129\": container with ID starting with 68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129 not found: ID does not exist" containerID="68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129"
Sep 11 23:23:25.578172 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.578112    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129"} err="failed to get container status \"68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129\": rpc error: code = NotFound desc = could not find container \"68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129\": container with ID starting with 68dca95193a0ac0dc022c4195e1d65347fd038815494aa15a8ccc40dbdd09129 not found: ID does not exist"
Sep 11 23:23:25.578172 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.578129    2580 scope.go:117] "RemoveContainer" containerID="a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233"
Sep 11 23:23:25.586128 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.586110    2580 scope.go:117] "RemoveContainer" containerID="a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233"
Sep 11 23:23:25.586418 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:23:25.586399    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233\": container with ID starting with a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233 not found: ID does not exist" containerID="a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233"
Sep 11 23:23:25.586498 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.586422    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233"} err="failed to get container status \"a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233\": rpc error: code = NotFound desc = could not find container \"a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233\": container with ID starting with a122321e670c89496bc30d27ad0a6338cad66d6d5677ef1aa9b932905f217233 not found: ID does not exist"
Sep 11 23:23:25.595132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.595110    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"]
Sep 11 23:23:25.601032 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.601013    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1ebfb31f-764b8d6c8f-7tshx"]
Sep 11 23:23:25.612760 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.612742    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"]
Sep 11 23:23:25.615122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.615103    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1ebfb31f-6747d7d554-m2vvn"]
Sep 11 23:23:25.643858 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:25.643837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:26.150153 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:26.150123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:26.858805 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:26.858770    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:27.160399 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:27.160298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:27.413948 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:27.413855    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" path="/var/lib/kubelet/pods/6bcdae1e-4f8d-4e78-8bea-322c327fe4fb/volumes"
Sep 11 23:23:27.414227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:27.414209    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd5ac1d6-9458-43c9-808e-995e80c51423" path="/var/lib/kubelet/pods/dd5ac1d6-9458-43c9-808e-995e80c51423/volumes"
Sep 11 23:23:27.752341 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:27.752253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:28.218009 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.217963    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc" podUID="5ac9a235-5bde-497f-90a7-6a7055e2e60b" containerName="main" containerID="cri-o://fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6" gracePeriod=45
Sep 11 23:23:28.374183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.374148    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc" podUID="5ac9a235-5bde-497f-90a7-6a7055e2e60b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": dial tcp 10.134.0.52:8000: connect: connection refused"
Sep 11 23:23:28.457562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.457531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:28.473403 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.473340    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:23:28.587182 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587152    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-dshm\") pod \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") "
Sep 11 23:23:28.587360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-model-cache\") pod \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") "
Sep 11 23:23:28.587360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587243    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tls-certs\") pod \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") "
Sep 11 23:23:28.587360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587267    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmkdq\" (UniqueName: \"kubernetes.io/projected/5ac9a235-5bde-497f-90a7-6a7055e2e60b-kube-api-access-nmkdq\") pod \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") "
Sep 11 23:23:28.587360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587297    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tmp-dir\") pod \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") "
Sep 11 23:23:28.587360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587329    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-home\") pod \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\" (UID: \"5ac9a235-5bde-497f-90a7-6a7055e2e60b\") "
Sep 11 23:23:28.587631 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587475    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ac9a235-5bde-497f-90a7-6a7055e2e60b" (UID: "5ac9a235-5bde-497f-90a7-6a7055e2e60b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:23:28.587687 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587633    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ac9a235-5bde-497f-90a7-6a7055e2e60b" (UID: "5ac9a235-5bde-497f-90a7-6a7055e2e60b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:23:28.587687 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587642    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:28.587687 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.587666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-home" (OuterVolumeSpecName: "home") pod "5ac9a235-5bde-497f-90a7-6a7055e2e60b" (UID: "5ac9a235-5bde-497f-90a7-6a7055e2e60b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:23:28.589685 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.589652    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac9a235-5bde-497f-90a7-6a7055e2e60b-kube-api-access-nmkdq" (OuterVolumeSpecName: "kube-api-access-nmkdq") pod "5ac9a235-5bde-497f-90a7-6a7055e2e60b" (UID: "5ac9a235-5bde-497f-90a7-6a7055e2e60b"). InnerVolumeSpecName "kube-api-access-nmkdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:23:28.589815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.589677    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ac9a235-5bde-497f-90a7-6a7055e2e60b" (UID: "5ac9a235-5bde-497f-90a7-6a7055e2e60b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:23:28.589815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.589722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-dshm" (OuterVolumeSpecName: "dshm") pod "5ac9a235-5bde-497f-90a7-6a7055e2e60b" (UID: "5ac9a235-5bde-497f-90a7-6a7055e2e60b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:23:28.589953 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.589898    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ac9a235-5bde-497f-90a7-6a7055e2e60b" containerID="fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6" exitCode=0
Sep 11 23:23:28.590017 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.589953    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"
Sep 11 23:23:28.590076 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.589944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc" event={"ID":"5ac9a235-5bde-497f-90a7-6a7055e2e60b","Type":"ContainerDied","Data":"fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6"}
Sep 11 23:23:28.590076 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.590059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc" event={"ID":"5ac9a235-5bde-497f-90a7-6a7055e2e60b","Type":"ContainerDied","Data":"31282a856553622bf4f997db92ff6dcbfa6a75f04232dfb280470eb6c4a2d3b5"}
Sep 11 23:23:28.590173 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.590077    2580 scope.go:117] "RemoveContainer" containerID="fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6"
Sep 11 23:23:28.607754 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.607739    2580 scope.go:117] "RemoveContainer" containerID="fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6"
Sep 11 23:23:28.608035 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:23:28.608016    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6\": container with ID starting with fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6 not found: ID does not exist" containerID="fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6"
Sep 11 23:23:28.608128 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.608044    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6"} err="failed to get container status \"fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6\": rpc error: code = NotFound desc = could not find container \"fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6\": container with ID starting with fe2d6b7c4422e8adb412402b42ebc73e9b6fce2af1fac20c1597f871ecb29bb6 not found: ID does not exist"
Sep 11 23:23:28.619532 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.619511    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"]
Sep 11 23:23:28.623141 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.623117    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-7719ed/explicit-model-7719ed-kserve-d4c77f79f-r44hc"]
Sep 11 23:23:28.688265 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.688239    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:28.688265 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.688260    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:28.688265 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.688269    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:28.688458 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.688280    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmkdq\" (UniqueName: \"kubernetes.io/projected/5ac9a235-5bde-497f-90a7-6a7055e2e60b-kube-api-access-nmkdq\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:28.688458 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:28.688290    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ac9a235-5bde-497f-90a7-6a7055e2e60b-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:23:29.356993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:29.356949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:29.413366 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:29.413335    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac9a235-5bde-497f-90a7-6a7055e2e60b" path="/var/lib/kubelet/pods/5ac9a235-5bde-497f-90a7-6a7055e2e60b/volumes"
Sep 11 23:23:31.477175 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:31.477137    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:31.575760 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:31.575730    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:31.675733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:31.675698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:31.762968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:31.762868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:32.233409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:32.233374    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:32.345969 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:32.345938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:33.715029 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:33.714986    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:33.959632 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:33.959598    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:37.962023 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:37.961989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:38.066091 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:38.066058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:38.162117 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:38.162082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:38.358361 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:38.358325    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:39.400337 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:39.400301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:39.703338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:39.703251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:39.921209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:39.921170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:41.096009 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:41.095973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:41.195245 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:41.195190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:41.307325 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:41.307290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:44.022877 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:44.022847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:44.093332 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:44.093304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:44.889071 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:44.889038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:45.189046 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:45.188952    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:45.503958 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:45.503860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:47.641643 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:47.641561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:47.749553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:47.749520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:47.846184 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:47.846150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:48.594788 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:48.594757    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:49.459469 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:49.459433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:50.163073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:50.163035    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:50.864744 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:50.864713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:50.966489 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:50.966448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:51.068128 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.068086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:51.353248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.353216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:51.454419 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.454389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:23:51.487294 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487257    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"]
Sep 11 23:23:51.487587 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487576    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd5ac1d6-9458-43c9-808e-995e80c51423" containerName="payload-pre-processing"
Sep 11 23:23:51.487636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487589    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd5ac1d6-9458-43c9-808e-995e80c51423" containerName="payload-pre-processing"
Sep 11 23:23:51.487636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487599    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" containerName="payload-processing"
Sep 11 23:23:51.487636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487605    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" containerName="payload-processing"
Sep 11 23:23:51.487636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487615    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" containerName="payload-processing"
Sep 11 23:23:51.487636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487621    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" containerName="payload-processing"
Sep 11 23:23:51.487636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487631    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac9a235-5bde-497f-90a7-6a7055e2e60b" containerName="main"
Sep 11 23:23:51.487636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487636    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac9a235-5bde-497f-90a7-6a7055e2e60b" containerName="main"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487647    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bfcd20-a3a9-452e-94fd-e7740a393ce3" containerName="payload-pre-processing"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487655    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bfcd20-a3a9-452e-94fd-e7740a393ce3" containerName="payload-pre-processing"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487663    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7fb448d-362b-459e-a3fd-55480bbbfbc6" containerName="revoke-keys"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487668    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7fb448d-362b-459e-a3fd-55480bbbfbc6" containerName="revoke-keys"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487722    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1e6fb17-dc5f-4b73-b8de-94bbf5a0ce68" containerName="payload-processing"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487731    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bfcd20-a3a9-452e-94fd-e7740a393ce3" containerName="payload-pre-processing"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487738    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7fb448d-362b-459e-a3fd-55480bbbfbc6" containerName="revoke-keys"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487744    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd5ac1d6-9458-43c9-808e-995e80c51423" containerName="payload-pre-processing"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487750    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bcdae1e-4f8d-4e78-8bea-322c327fe4fb" containerName="payload-processing"
Sep 11 23:23:51.487849 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.487755    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac9a235-5bde-497f-90a7-6a7055e2e60b" containerName="main"
Sep 11 23:23:51.493810 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.493789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.496498 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.496475    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-cd48eb\""
Sep 11 23:23:51.503943 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.503907    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"]
Sep 11 23:23:51.505596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.505576    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"]
Sep 11 23:23:51.508954 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.508937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:51.511721 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.511701    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-cd48eb-dockercfg-94vmw\""
Sep 11 23:23:51.519842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.519821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"]
Sep 11 23:23:51.680008 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.679907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/890bb418-26ac-4a5a-9abe-99de37cbccf3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.680008 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.679976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjkvw\" (UniqueName: \"kubernetes.io/projected/890bb418-26ac-4a5a-9abe-99de37cbccf3-kube-api-access-sjkvw\") pod \"payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.680008 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.680007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm7dh\" (UniqueName: \"kubernetes.io/projected/de2d5069-c1f7-4362-a802-ece4efcfbf3b-kube-api-access-nm7dh\") pod \"payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:51.680226 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.680129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de2d5069-c1f7-4362-a802-ece4efcfbf3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:51.781239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.781206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjkvw\" (UniqueName: \"kubernetes.io/projected/890bb418-26ac-4a5a-9abe-99de37cbccf3-kube-api-access-sjkvw\") pod \"payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.781450 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.781255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm7dh\" (UniqueName: \"kubernetes.io/projected/de2d5069-c1f7-4362-a802-ece4efcfbf3b-kube-api-access-nm7dh\") pod \"payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:51.781450 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.781314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de2d5069-c1f7-4362-a802-ece4efcfbf3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:51.781450 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.781367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/890bb418-26ac-4a5a-9abe-99de37cbccf3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.782052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.782034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/890bb418-26ac-4a5a-9abe-99de37cbccf3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.782052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.782044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de2d5069-c1f7-4362-a802-ece4efcfbf3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:51.789558 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.789528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjkvw\" (UniqueName: \"kubernetes.io/projected/890bb418-26ac-4a5a-9abe-99de37cbccf3-kube-api-access-sjkvw\") pod \"payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.789668 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.789597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm7dh\" (UniqueName: \"kubernetes.io/projected/de2d5069-c1f7-4362-a802-ece4efcfbf3b-kube-api-access-nm7dh\") pod \"payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:51.804961 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.804942    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:51.819064 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:51.819043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:52.078415 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.078389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"]
Sep 11 23:23:52.080667 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:23:52.080636    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890bb418_26ac_4a5a_9abe_99de37cbccf3.slice/crio-e95e29fd863b352d36429ee4bbca9aa01558e8a6cef858507293ad2b2bfbd01e WatchSource:0}: Error finding container e95e29fd863b352d36429ee4bbca9aa01558e8a6cef858507293ad2b2bfbd01e: Status 404 returned error can't find the container with id e95e29fd863b352d36429ee4bbca9aa01558e8a6cef858507293ad2b2bfbd01e
Sep 11 23:23:52.099899 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.099878    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"]
Sep 11 23:23:52.102439 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:23:52.102416    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2d5069_c1f7_4362_a802_ece4efcfbf3b.slice/crio-ede1a07280d9041830568e4fc2c8dd1478f0dfa4490e7f73374107e7bf8ffea0 WatchSource:0}: Error finding container ede1a07280d9041830568e4fc2c8dd1478f0dfa4490e7f73374107e7bf8ffea0: Status 404 returned error can't find the container with id ede1a07280d9041830568e4fc2c8dd1478f0dfa4490e7f73374107e7bf8ffea0
Sep 11 23:23:52.705348 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.705314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx" event={"ID":"de2d5069-c1f7-4362-a802-ece4efcfbf3b","Type":"ContainerStarted","Data":"153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae"}
Sep 11 23:23:52.705846 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.705381    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:23:52.705846 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.705407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx" event={"ID":"de2d5069-c1f7-4362-a802-ece4efcfbf3b","Type":"ContainerStarted","Data":"ede1a07280d9041830568e4fc2c8dd1478f0dfa4490e7f73374107e7bf8ffea0"}
Sep 11 23:23:52.707130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.707106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4" event={"ID":"890bb418-26ac-4a5a-9abe-99de37cbccf3","Type":"ContainerStarted","Data":"1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b"}
Sep 11 23:23:52.707241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.707152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4" event={"ID":"890bb418-26ac-4a5a-9abe-99de37cbccf3","Type":"ContainerStarted","Data":"e95e29fd863b352d36429ee4bbca9aa01558e8a6cef858507293ad2b2bfbd01e"}
Sep 11 23:23:52.707241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.707188    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:23:52.722827 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.722772    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx" podStartSLOduration=1.722754449 podStartE2EDuration="1.722754449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:23:52.722606681 +0000 UTC m=+1347.934577857" watchObservedRunningTime="2026-09-11 23:23:52.722754449 +0000 UTC m=+1347.934725624"
Sep 11 23:23:52.740141 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:52.740103    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4" podStartSLOduration=1.740088912 podStartE2EDuration="1.740088912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:23:52.738641664 +0000 UTC m=+1347.950612839" watchObservedRunningTime="2026-09-11 23:23:52.740088912 +0000 UTC m=+1347.952060085"
Sep 11 23:23:53.883109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:23:53.883072    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:03.713881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:03.713846    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:24:03.714345 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:03.714213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:24:03.758312 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:03.758281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:11.775685 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:11.775647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:13.229607 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:13.229566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:13.950550 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:13.950498    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:14.651746 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:14.651703    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:15.467996 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:15.467964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:16.210580 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:16.210548    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:16.953038 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:16.952998    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:17.704600 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:17.704554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:18.012468 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:18.012356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:18.328239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:18.328202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:18.653434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:18.653346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:20.273531 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:20.273490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:21.165346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:21.165300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:21.260965 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:21.260892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:22.058388 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:22.058348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:23.058733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:23.058702    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:23.848410 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:23.848381    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:24.545935 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:24.545879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:25.478622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:25.478588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:25.579793 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:25.579764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:25.953639 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:25.953609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:26.054133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.054102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:26.065723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.065695    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"]
Sep 11 23:24:26.070440 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.070420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.073282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.073260    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-97d9dc\""
Sep 11 23:24:26.073382 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.073269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-97d9dc-dockercfg-5jcn8\""
Sep 11 23:24:26.080937 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.080885    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"]
Sep 11 23:24:26.099201 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.099174    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"]
Sep 11 23:24:26.103183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.103162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.112868 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.112848    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"]
Sep 11 23:24:26.128924 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.128873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:26.197438 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.197404    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.197562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.197444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159859-d10a-4267-b5cc-05ceef02a4bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.197562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.197510    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhxxd\" (UniqueName: \"kubernetes.io/projected/80159859-d10a-4267-b5cc-05ceef02a4bc-kube-api-access-hhxxd\") pod \"payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.197660 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.197558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx4mx\" (UniqueName: \"kubernetes.io/projected/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-kube-api-access-xx4mx\") pod \"payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.298876 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.298803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhxxd\" (UniqueName: \"kubernetes.io/projected/80159859-d10a-4267-b5cc-05ceef02a4bc-kube-api-access-hhxxd\") pod \"payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.298876 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.298843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx4mx\" (UniqueName: \"kubernetes.io/projected/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-kube-api-access-xx4mx\") pod \"payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.299082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.298941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.299082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.298971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159859-d10a-4267-b5cc-05ceef02a4bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.299701 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.299680    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159859-d10a-4267-b5cc-05ceef02a4bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.299754 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.299682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.306959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.306935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx4mx\" (UniqueName: \"kubernetes.io/projected/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-kube-api-access-xx4mx\") pod \"payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.307049 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.307001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhxxd\" (UniqueName: \"kubernetes.io/projected/80159859-d10a-4267-b5cc-05ceef02a4bc-kube-api-access-hhxxd\") pod \"payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.382006 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.381974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.412799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.412775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.653091 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.653065    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"]
Sep 11 23:24:26.654870 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:24:26.654845    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80159859_d10a_4267_b5cc_05ceef02a4bc.slice/crio-7116f85322d8391c35ef387a9dd752cc3a164839a1bfbdc56759cc86ed5333b2 WatchSource:0}: Error finding container 7116f85322d8391c35ef387a9dd752cc3a164839a1bfbdc56759cc86ed5333b2: Status 404 returned error can't find the container with id 7116f85322d8391c35ef387a9dd752cc3a164839a1bfbdc56759cc86ed5333b2
Sep 11 23:24:26.692987 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.692963    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"]
Sep 11 23:24:26.694575 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:24:26.694553    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a01628_25d7_44bc_9096_8e1e3ee5aa7b.slice/crio-7c8b9ebf89ba4676632fa13dc19d299c65656fd61cd39e8a8ee93f8bdbab205d WatchSource:0}: Error finding container 7c8b9ebf89ba4676632fa13dc19d299c65656fd61cd39e8a8ee93f8bdbab205d: Status 404 returned error can't find the container with id 7c8b9ebf89ba4676632fa13dc19d299c65656fd61cd39e8a8ee93f8bdbab205d
Sep 11 23:24:26.868526 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.868491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x" event={"ID":"21a01628-25d7-44bc-9096-8e1e3ee5aa7b","Type":"ContainerStarted","Data":"2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68"}
Sep 11 23:24:26.868755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.868559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x" event={"ID":"21a01628-25d7-44bc-9096-8e1e3ee5aa7b","Type":"ContainerStarted","Data":"7c8b9ebf89ba4676632fa13dc19d299c65656fd61cd39e8a8ee93f8bdbab205d"}
Sep 11 23:24:26.868755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.868599    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:26.870245 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.870219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6" event={"ID":"80159859-d10a-4267-b5cc-05ceef02a4bc","Type":"ContainerStarted","Data":"ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5"}
Sep 11 23:24:26.870352 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.870273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6" event={"ID":"80159859-d10a-4267-b5cc-05ceef02a4bc","Type":"ContainerStarted","Data":"7116f85322d8391c35ef387a9dd752cc3a164839a1bfbdc56759cc86ed5333b2"}
Sep 11 23:24:26.870352 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.870301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:26.885462 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.885407    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x" podStartSLOduration=0.88538763 podStartE2EDuration="885.38763ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:26.884351203 +0000 UTC m=+1382.096322377" watchObservedRunningTime="2026-09-11 23:24:26.88538763 +0000 UTC m=+1382.097358808"
Sep 11 23:24:26.900818 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:26.900772    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6" podStartSLOduration=0.90075908 podStartE2EDuration="900.75908ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:26.899271565 +0000 UTC m=+1382.111242741" watchObservedRunningTime="2026-09-11 23:24:26.90075908 +0000 UTC m=+1382.112730254"
Sep 11 23:24:37.876964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:37.876935    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:24:37.877343 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:37.877002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:24:47.575712 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:47.575675    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:48.361700 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:48.361664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:49.218381 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.218352    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:49.623050 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.623019    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"]
Sep 11 23:24:49.627455 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.627433    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.632339 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.632319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-c0d4bf\"/\"default-dockercfg-z8sd5\""
Sep 11 23:24:49.632530 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.632319    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-c0d4bf\"/\"openshift-service-ca.crt\""
Sep 11 23:24:49.632626 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.632403    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-c0d4bf\"/\"kube-root-ca.crt\""
Sep 11 23:24:49.632999 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.632981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-c0d4bf\"/\"noten-model-c0d4bf-kserve-self-signed-certs\""
Sep 11 23:24:49.637409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.637387    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"]
Sep 11 23:24:49.657467 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.657441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:49.687291 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.687249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-model-cache\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.687398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.687324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-home\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.687398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.687355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-dshm\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.687398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.687379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-tmp-dir\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.687557 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.687458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84599859-20e2-4aca-9398-35671efbef7d-tls-certs\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.687557 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.687543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c4dw\" (UniqueName: \"kubernetes.io/projected/84599859-20e2-4aca-9398-35671efbef7d-kube-api-access-6c4dw\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.788634 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.788592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-model-cache\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.788839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.788668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-home\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.788839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.788713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-dshm\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.788839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.788744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-tmp-dir\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.788839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.788776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84599859-20e2-4aca-9398-35671efbef7d-tls-certs\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.789097 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.788973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c4dw\" (UniqueName: \"kubernetes.io/projected/84599859-20e2-4aca-9398-35671efbef7d-kube-api-access-6c4dw\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.789157 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.789091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-model-cache\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.789157 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.789100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-home\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.789157 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.789147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-tmp-dir\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.791114 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.791093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-dshm\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.791423 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.791405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84599859-20e2-4aca-9398-35671efbef7d-tls-certs\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.797348 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.797326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c4dw\" (UniqueName: \"kubernetes.io/projected/84599859-20e2-4aca-9398-35671efbef7d-kube-api-access-6c4dw\") pod \"noten-model-c0d4bf-kserve-6bf6f89699-hmdwd\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") " pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.940153 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.940068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:49.950279 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:49.950249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:50.216417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:50.216340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"]
Sep 11 23:24:50.219620 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:24:50.219593    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84599859_20e2_4aca_9398_35671efbef7d.slice/crio-9b56cae4d5d8cc6ec7978f6c352a3f3b09c21694b0a73a5d8eab19c6a3c114b0 WatchSource:0}: Error finding container 9b56cae4d5d8cc6ec7978f6c352a3f3b09c21694b0a73a5d8eab19c6a3c114b0: Status 404 returned error can't find the container with id 9b56cae4d5d8cc6ec7978f6c352a3f3b09c21694b0a73a5d8eab19c6a3c114b0
Sep 11 23:24:50.767331 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:50.767296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:50.975100 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:50.975052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:50.989883 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:50.989853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd" event={"ID":"84599859-20e2-4aca-9398-35671efbef7d","Type":"ContainerStarted","Data":"1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444"}
Sep 11 23:24:50.990040 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:50.989904    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd" event={"ID":"84599859-20e2-4aca-9398-35671efbef7d","Type":"ContainerStarted","Data":"9b56cae4d5d8cc6ec7978f6c352a3f3b09c21694b0a73a5d8eab19c6a3c114b0"}
Sep 11 23:24:51.010514 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.010468    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd" podStartSLOduration=2.010454921 podStartE2EDuration="2.010454921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:51.009295833 +0000 UTC m=+1406.221267007" watchObservedRunningTime="2026-09-11 23:24:51.010454921 +0000 UTC m=+1406.222426095"
Sep 11 23:24:51.613359 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.613324    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:51.673377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.673347    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7604-97b51b"]
Sep 11 23:24:51.676735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.676716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:51.679841 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.679824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8cxx9\""
Sep 11 23:24:51.682485 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.682461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-97b51b"]
Sep 11 23:24:51.704869 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.704847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7lbr\" (UniqueName: \"kubernetes.io/projected/8b96e991-3354-4aed-b480-e07b0832b8cb-kube-api-access-k7lbr\") pod \"test-curl-7604-97b51b\" (UID: \"8b96e991-3354-4aed-b480-e07b0832b8cb\") " pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:51.806208 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.806179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7lbr\" (UniqueName: \"kubernetes.io/projected/8b96e991-3354-4aed-b480-e07b0832b8cb-kube-api-access-k7lbr\") pod \"test-curl-7604-97b51b\" (UID: \"8b96e991-3354-4aed-b480-e07b0832b8cb\") " pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:51.816521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.816499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7lbr\" (UniqueName: \"kubernetes.io/projected/8b96e991-3354-4aed-b480-e07b0832b8cb-kube-api-access-k7lbr\") pod \"test-curl-7604-97b51b\" (UID: \"8b96e991-3354-4aed-b480-e07b0832b8cb\") " pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:51.987962 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:51.987871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:52.258856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:52.258834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-97b51b"]
Sep 11 23:24:52.261090 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:24:52.261063    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b96e991_3354_4aed_b480_e07b0832b8cb.slice/crio-9295f19212e7f26aab976986f6cea2ea266fb1bbc487ad77b87df6ddeffcd502 WatchSource:0}: Error finding container 9295f19212e7f26aab976986f6cea2ea266fb1bbc487ad77b87df6ddeffcd502: Status 404 returned error can't find the container with id 9295f19212e7f26aab976986f6cea2ea266fb1bbc487ad77b87df6ddeffcd502
Sep 11 23:24:52.357223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:52.357198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:52.924318 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:52.924283    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"]
Sep 11 23:24:52.947407 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:52.947375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:53.001688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.001651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-97b51b" event={"ID":"8b96e991-3354-4aed-b480-e07b0832b8cb","Type":"ContainerStarted","Data":"d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272"}
Sep 11 23:24:53.001688 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.001693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-97b51b" event={"ID":"8b96e991-3354-4aed-b480-e07b0832b8cb","Type":"ContainerStarted","Data":"9295f19212e7f26aab976986f6cea2ea266fb1bbc487ad77b87df6ddeffcd502"}
Sep 11 23:24:53.017479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.017432    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7604-97b51b" podStartSLOduration=2.017416777 podStartE2EDuration="2.017416777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:53.017007796 +0000 UTC m=+1408.228978970" watchObservedRunningTime="2026-09-11 23:24:53.017416777 +0000 UTC m=+1408.229387955"
Sep 11 23:24:53.360866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.360831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:53.575087 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.575054    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7604-97b51b"]
Sep 11 23:24:53.577816 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.577788    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7604-97b51b"]
Sep 11 23:24:53.689118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.689037    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7604-0c8dbc"]
Sep 11 23:24:53.689429 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.689417    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" containerName="test-curl-7604-97b51b"
Sep 11 23:24:53.689516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.689431    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" containerName="test-curl-7604-97b51b"
Sep 11 23:24:53.689516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.689487    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" containerName="test-curl-7604-97b51b"
Sep 11 23:24:53.692463 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.692444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:53.696951 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.696906    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-0c8dbc"]
Sep 11 23:24:53.722453 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.722419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brcxg\" (UniqueName: \"kubernetes.io/projected/b64656bb-1de5-40ac-9a6b-203ea5960083-kube-api-access-brcxg\") pod \"test-curl-7604-0c8dbc\" (UID: \"b64656bb-1de5-40ac-9a6b-203ea5960083\") " pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:53.823205 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.823176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brcxg\" (UniqueName: \"kubernetes.io/projected/b64656bb-1de5-40ac-9a6b-203ea5960083-kube-api-access-brcxg\") pod \"test-curl-7604-0c8dbc\" (UID: \"b64656bb-1de5-40ac-9a6b-203ea5960083\") " pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:53.832254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:53.832232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brcxg\" (UniqueName: \"kubernetes.io/projected/b64656bb-1de5-40ac-9a6b-203ea5960083-kube-api-access-brcxg\") pod \"test-curl-7604-0c8dbc\" (UID: \"b64656bb-1de5-40ac-9a6b-203ea5960083\") " pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:54.003843 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:54.003765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:54.006963 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:54.006942    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7604-97b51b" container="test-curl-7604-97b51b"
Sep 11 23:24:54.088470 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:54.088446    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:54.273087 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:54.272972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-0c8dbc"]
Sep 11 23:24:54.275447 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:24:54.275419    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb64656bb_1de5_40ac_9a6b_203ea5960083.slice/crio-f53311ea991baa8964997a03defe85d7711ae8429e90853d68dd67ab663af98b WatchSource:0}: Error finding container f53311ea991baa8964997a03defe85d7711ae8429e90853d68dd67ab663af98b: Status 404 returned error can't find the container with id f53311ea991baa8964997a03defe85d7711ae8429e90853d68dd67ab663af98b
Sep 11 23:24:54.516778 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:54.516746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:54.796043 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:54.796012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:55.013457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.013421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-0c8dbc" event={"ID":"b64656bb-1de5-40ac-9a6b-203ea5960083","Type":"ContainerStarted","Data":"c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd"}
Sep 11 23:24:55.013881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.013464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-0c8dbc" event={"ID":"b64656bb-1de5-40ac-9a6b-203ea5960083","Type":"ContainerStarted","Data":"f53311ea991baa8964997a03defe85d7711ae8429e90853d68dd67ab663af98b"}
Sep 11 23:24:55.013881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.013531    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7604-97b51b" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" containerName="test-curl-7604-97b51b" containerID="cri-o://d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272" gracePeriod=2
Sep 11 23:24:55.016678 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:55.016655    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-97b51b\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:55.033937 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.033882    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7604-0c8dbc" podStartSLOduration=2.033868671 podStartE2EDuration="2.033868671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:55.032476848 +0000 UTC m=+1410.244448024" watchObservedRunningTime="2026-09-11 23:24:55.033868671 +0000 UTC m=+1410.245839845"
Sep 11 23:24:55.035038 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:55.035015    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-97b51b\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:55.416156 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:55.416116    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-97b51b\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:55.560987 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.560887    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:55.590373 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.590341    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7604-0c8dbc"]
Sep 11 23:24:55.593687 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.593662    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7604-0c8dbc"]
Sep 11 23:24:55.701267 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.701241    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7604-897ccd"]
Sep 11 23:24:55.701660 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.701646    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" containerName="test-curl-7604-0c8dbc"
Sep 11 23:24:55.701732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.701662    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" containerName="test-curl-7604-0c8dbc"
Sep 11 23:24:55.701777 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.701757    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" containerName="test-curl-7604-0c8dbc"
Sep 11 23:24:55.704860 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.704843    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:24:55.711752 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.711725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-897ccd"]
Sep 11 23:24:55.739088 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.739058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq54t\" (UniqueName: \"kubernetes.io/projected/0af32d74-0bf9-49af-8a16-b1265292e975-kube-api-access-xq54t\") pod \"test-curl-7604-897ccd\" (UID: \"0af32d74-0bf9-49af-8a16-b1265292e975\") " pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:24:55.840183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.840154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq54t\" (UniqueName: \"kubernetes.io/projected/0af32d74-0bf9-49af-8a16-b1265292e975-kube-api-access-xq54t\") pod \"test-curl-7604-897ccd\" (UID: \"0af32d74-0bf9-49af-8a16-b1265292e975\") " pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:24:55.849031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:55.849006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq54t\" (UniqueName: \"kubernetes.io/projected/0af32d74-0bf9-49af-8a16-b1265292e975-kube-api-access-xq54t\") pod \"test-curl-7604-897ccd\" (UID: \"0af32d74-0bf9-49af-8a16-b1265292e975\") " pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:24:56.017113 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:56.017092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:24:56.018357 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:56.018338    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7604-0c8dbc" container="test-curl-7604-0c8dbc"
Sep 11 23:24:56.284453 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:56.284430    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-897ccd"]
Sep 11 23:24:56.286904 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:24:56.286867    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0af32d74_0bf9_49af_8a16_b1265292e975.slice/crio-fea2878933809d3963bcbe18edaf55115f16a544e966513b629383760c8a3776 WatchSource:0}: Error finding container fea2878933809d3963bcbe18edaf55115f16a544e966513b629383760c8a3776: Status 404 returned error can't find the container with id fea2878933809d3963bcbe18edaf55115f16a544e966513b629383760c8a3776
Sep 11 23:24:56.344094 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:56.344068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:57.025200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.025164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-897ccd" event={"ID":"0af32d74-0bf9-49af-8a16-b1265292e975","Type":"ContainerStarted","Data":"e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c"}
Sep 11 23:24:57.025624 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.025213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-897ccd" event={"ID":"0af32d74-0bf9-49af-8a16-b1265292e975","Type":"ContainerStarted","Data":"fea2878933809d3963bcbe18edaf55115f16a544e966513b629383760c8a3776"}
Sep 11 23:24:57.025624 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.025231    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7604-0c8dbc" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" containerName="test-curl-7604-0c8dbc" containerID="cri-o://c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd" gracePeriod=2
Sep 11 23:24:57.041132 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.041087    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7604-897ccd" podStartSLOduration=2.041071322 podStartE2EDuration="2.041071322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:57.039985649 +0000 UTC m=+1412.251956823" watchObservedRunningTime="2026-09-11 23:24:57.041071322 +0000 UTC m=+1412.253042495"
Sep 11 23:24:57.042206 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:57.042180    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-0c8dbc\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:57.044423 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:57.044395    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-0c8dbc\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:57.147302 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.147276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:57.159386 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.159366    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:57.161676 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:57.161653    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-0c8dbc\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:57.252674 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.252650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7lbr\" (UniqueName: \"kubernetes.io/projected/8b96e991-3354-4aed-b480-e07b0832b8cb-kube-api-access-k7lbr\") pod \"8b96e991-3354-4aed-b480-e07b0832b8cb\" (UID: \"8b96e991-3354-4aed-b480-e07b0832b8cb\") "
Sep 11 23:24:57.254509 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.254483    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b96e991-3354-4aed-b480-e07b0832b8cb-kube-api-access-k7lbr" (OuterVolumeSpecName: "kube-api-access-k7lbr") pod "8b96e991-3354-4aed-b480-e07b0832b8cb" (UID: "8b96e991-3354-4aed-b480-e07b0832b8cb"). InnerVolumeSpecName "kube-api-access-k7lbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:24:57.353347 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.353321    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7lbr\" (UniqueName: \"kubernetes.io/projected/8b96e991-3354-4aed-b480-e07b0832b8cb-kube-api-access-k7lbr\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:24:57.413262 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.413234    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b96e991-3354-4aed-b480-e07b0832b8cb" path="/var/lib/kubelet/pods/8b96e991-3354-4aed-b480-e07b0832b8cb/volumes"
Sep 11 23:24:57.598218 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.598183    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7604-897ccd"]
Sep 11 23:24:57.601011 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.600977    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7604-897ccd"]
Sep 11 23:24:57.727440 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.727364    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7604-66f6f4"]
Sep 11 23:24:57.727791 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.727772    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af32d74-0bf9-49af-8a16-b1265292e975" containerName="test-curl-7604-897ccd"
Sep 11 23:24:57.727791 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.727791    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af32d74-0bf9-49af-8a16-b1265292e975" containerName="test-curl-7604-897ccd"
Sep 11 23:24:57.727940 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.727870    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af32d74-0bf9-49af-8a16-b1265292e975" containerName="test-curl-7604-897ccd"
Sep 11 23:24:57.731054 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.731036    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:24:57.736288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.736264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-66f6f4"]
Sep 11 23:24:57.757288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.757262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pszhb\" (UniqueName: \"kubernetes.io/projected/217793f3-46d7-4840-92bc-5fc40a8a174e-kube-api-access-pszhb\") pod \"test-curl-7604-66f6f4\" (UID: \"217793f3-46d7-4840-92bc-5fc40a8a174e\") " pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:24:57.846390 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.846361    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:24:57.858481 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.858457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pszhb\" (UniqueName: \"kubernetes.io/projected/217793f3-46d7-4840-92bc-5fc40a8a174e-kube-api-access-pszhb\") pod \"test-curl-7604-66f6f4\" (UID: \"217793f3-46d7-4840-92bc-5fc40a8a174e\") " pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:24:57.867005 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:57.866979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pszhb\" (UniqueName: \"kubernetes.io/projected/217793f3-46d7-4840-92bc-5fc40a8a174e-kube-api-access-pszhb\") pod \"test-curl-7604-66f6f4\" (UID: \"217793f3-46d7-4840-92bc-5fc40a8a174e\") " pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:24:58.037288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:58.037195    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b96e991-3354-4aed-b480-e07b0832b8cb" containerID="d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272" exitCode=137
Sep 11 23:24:58.037288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:58.037241    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-97b51b"
Sep 11 23:24:58.037749 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:58.037301    2580 scope.go:117] "RemoveContainer" containerID="d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272"
Sep 11 23:24:58.037749 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:58.037497    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7604-897ccd" container="test-curl-7604-897ccd"
Sep 11 23:24:58.042211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:58.042193    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:24:58.048282 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:58.048265    2580 scope.go:117] "RemoveContainer" containerID="d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272"
Sep 11 23:24:58.048600 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:58.048574    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272\": container with ID starting with d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272 not found: ID does not exist" containerID="d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272"
Sep 11 23:24:58.048672 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:58.048613    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272"} err="failed to get container status \"d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272\": rpc error: code = NotFound desc = could not find container \"d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272\": container with ID starting with d9933628f82bf80d1396208f0076fcf32405172412db33ff12de980a66b9c272 not found: ID does not exist"
Sep 11 23:24:58.313886 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:58.313859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-66f6f4"]
Sep 11 23:24:58.316278 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:24:58.316248    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod217793f3_46d7_4840_92bc_5fc40a8a174e.slice/crio-00e75b3ac5a6dd5ca680a3140e7870b4cdd823292855fdaf1e6e7fa828026934 WatchSource:0}: Error finding container 00e75b3ac5a6dd5ca680a3140e7870b4cdd823292855fdaf1e6e7fa828026934: Status 404 returned error can't find the container with id 00e75b3ac5a6dd5ca680a3140e7870b4cdd823292855fdaf1e6e7fa828026934
Sep 11 23:24:59.043349 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.043315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-66f6f4" event={"ID":"217793f3-46d7-4840-92bc-5fc40a8a174e","Type":"ContainerStarted","Data":"0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659"}
Sep 11 23:24:59.043718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.043371    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-66f6f4" event={"ID":"217793f3-46d7-4840-92bc-5fc40a8a174e","Type":"ContainerStarted","Data":"00e75b3ac5a6dd5ca680a3140e7870b4cdd823292855fdaf1e6e7fa828026934"}
Sep 11 23:24:59.044356 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.044333    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7604-897ccd" podUID="0af32d74-0bf9-49af-8a16-b1265292e975" containerName="test-curl-7604-897ccd" containerID="cri-o://e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c" gracePeriod=2
Sep 11 23:24:59.061544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.061501    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7604-66f6f4" podStartSLOduration=2.061486517 podStartE2EDuration="2.061486517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:59.059136979 +0000 UTC m=+1414.271108153" watchObservedRunningTime="2026-09-11 23:24:59.061486517 +0000 UTC m=+1414.273457724"
Sep 11 23:24:59.061963 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:59.061927    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-897ccd\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="0af32d74-0bf9-49af-8a16-b1265292e975" pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:24:59.189765 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.189741    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:24:59.192055 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:24:59.192028    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-897ccd\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="0af32d74-0bf9-49af-8a16-b1265292e975" pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:24:59.269710 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.269688    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brcxg\" (UniqueName: \"kubernetes.io/projected/b64656bb-1de5-40ac-9a6b-203ea5960083-kube-api-access-brcxg\") pod \"b64656bb-1de5-40ac-9a6b-203ea5960083\" (UID: \"b64656bb-1de5-40ac-9a6b-203ea5960083\") "
Sep 11 23:24:59.271489 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.271456    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64656bb-1de5-40ac-9a6b-203ea5960083-kube-api-access-brcxg" (OuterVolumeSpecName: "kube-api-access-brcxg") pod "b64656bb-1de5-40ac-9a6b-203ea5960083" (UID: "b64656bb-1de5-40ac-9a6b-203ea5960083"). InnerVolumeSpecName "kube-api-access-brcxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:24:59.370459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.370435    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brcxg\" (UniqueName: \"kubernetes.io/projected/b64656bb-1de5-40ac-9a6b-203ea5960083-kube-api-access-brcxg\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:24:59.414382 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.414355    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b64656bb-1de5-40ac-9a6b-203ea5960083" path="/var/lib/kubelet/pods/b64656bb-1de5-40ac-9a6b-203ea5960083/volumes"
Sep 11 23:24:59.629828 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.629802    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7604-66f6f4"]
Sep 11 23:24:59.634214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.634185    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7604-66f6f4"]
Sep 11 23:24:59.940718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.940634    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:59.940879 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.940776    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:24:59.953500 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:24:59.953476    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:25:00.050582 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.050548    2580 generic.go:356] "Generic (PLEG): container finished" podID="b64656bb-1de5-40ac-9a6b-203ea5960083" containerID="c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd" exitCode=137
Sep 11 23:25:00.051075 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.050589    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-0c8dbc"
Sep 11 23:25:00.051075 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.050654    2580 scope.go:117] "RemoveContainer" containerID="c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd"
Sep 11 23:25:00.051075 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:00.050939    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7604-66f6f4" container="test-curl-7604-66f6f4"
Sep 11 23:25:00.059399 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.059375    2580 scope.go:117] "RemoveContainer" containerID="c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd"
Sep 11 23:25:00.059684 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:00.059657    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd\": container with ID starting with c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd not found: ID does not exist" containerID="c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd"
Sep 11 23:25:00.059747 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.059696    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd"} err="failed to get container status \"c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd\": rpc error: code = NotFound desc = could not find container \"c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd\": container with ID starting with c23036af1d0180cbc760d87c66e47a665434f0ac0746728f99da1eb0daf9addd not found: ID does not exist"
Sep 11 23:25:00.062525 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.062507    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:25:00.208719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.208621    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:00.579085 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.579050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:00.895867 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:00.895779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:01.056429 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:01.056397    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7604-66f6f4" podUID="217793f3-46d7-4840-92bc-5fc40a8a174e" containerName="test-curl-7604-66f6f4" containerID="cri-o://0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659" gracePeriod=2
Sep 11 23:25:01.059108 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:01.059083    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-66f6f4\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="217793f3-46d7-4840-92bc-5fc40a8a174e" pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:25:01.218701 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:01.218678    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:25:01.221759 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:01.221713    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-66f6f4\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="217793f3-46d7-4840-92bc-5fc40a8a174e" pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:25:01.289048 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:01.289021    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq54t\" (UniqueName: \"kubernetes.io/projected/0af32d74-0bf9-49af-8a16-b1265292e975-kube-api-access-xq54t\") pod \"0af32d74-0bf9-49af-8a16-b1265292e975\" (UID: \"0af32d74-0bf9-49af-8a16-b1265292e975\") "
Sep 11 23:25:01.291039 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:01.291001    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af32d74-0bf9-49af-8a16-b1265292e975-kube-api-access-xq54t" (OuterVolumeSpecName: "kube-api-access-xq54t") pod "0af32d74-0bf9-49af-8a16-b1265292e975" (UID: "0af32d74-0bf9-49af-8a16-b1265292e975"). InnerVolumeSpecName "kube-api-access-xq54t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:01.390512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:01.390479    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq54t\" (UniqueName: \"kubernetes.io/projected/0af32d74-0bf9-49af-8a16-b1265292e975-kube-api-access-xq54t\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:01.414943 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:01.414842    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af32d74-0bf9-49af-8a16-b1265292e975" path="/var/lib/kubelet/pods/0af32d74-0bf9-49af-8a16-b1265292e975/volumes"
Sep 11 23:25:02.062524 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:02.062491    2580 generic.go:356] "Generic (PLEG): container finished" podID="0af32d74-0bf9-49af-8a16-b1265292e975" containerID="e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c" exitCode=137
Sep 11 23:25:02.062946 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:02.062533    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-897ccd"
Sep 11 23:25:02.062946 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:02.062601    2580 scope.go:117] "RemoveContainer" containerID="e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c"
Sep 11 23:25:02.071201 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:02.071183    2580 scope.go:117] "RemoveContainer" containerID="e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c"
Sep 11 23:25:02.071489 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:02.071469    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c\": container with ID starting with e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c not found: ID does not exist" containerID="e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c"
Sep 11 23:25:02.071558 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:02.071501    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c"} err="failed to get container status \"e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c\": rpc error: code = NotFound desc = could not find container \"e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c\": container with ID starting with e4d058130307b1b27e4a0bea1f8d8559491cf9b219479c227bde60db3ee44c1c not found: ID does not exist"
Sep 11 23:25:02.575518 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:02.575480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:02.655414 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:02.655369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:03.202578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.202550    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:25:03.309119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.309087    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pszhb\" (UniqueName: \"kubernetes.io/projected/217793f3-46d7-4840-92bc-5fc40a8a174e-kube-api-access-pszhb\") pod \"217793f3-46d7-4840-92bc-5fc40a8a174e\" (UID: \"217793f3-46d7-4840-92bc-5fc40a8a174e\") "
Sep 11 23:25:03.311105 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.311047    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/217793f3-46d7-4840-92bc-5fc40a8a174e-kube-api-access-pszhb" (OuterVolumeSpecName: "kube-api-access-pszhb") pod "217793f3-46d7-4840-92bc-5fc40a8a174e" (UID: "217793f3-46d7-4840-92bc-5fc40a8a174e"). InnerVolumeSpecName "kube-api-access-pszhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:03.317615 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.317590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:03.410206 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.410178    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pszhb\" (UniqueName: \"kubernetes.io/projected/217793f3-46d7-4840-92bc-5fc40a8a174e-kube-api-access-pszhb\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:03.414234 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.414212    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="217793f3-46d7-4840-92bc-5fc40a8a174e" path="/var/lib/kubelet/pods/217793f3-46d7-4840-92bc-5fc40a8a174e/volumes"
Sep 11 23:25:03.416267 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.416240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:03.517406 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:03.517377    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:04.073981 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:04.073948    2580 generic.go:356] "Generic (PLEG): container finished" podID="217793f3-46d7-4840-92bc-5fc40a8a174e" containerID="0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659" exitCode=137
Sep 11 23:25:04.074158 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:04.073997    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-66f6f4"
Sep 11 23:25:04.074158 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:04.074040    2580 scope.go:117] "RemoveContainer" containerID="0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659"
Sep 11 23:25:04.082866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:04.082723    2580 scope.go:117] "RemoveContainer" containerID="0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659"
Sep 11 23:25:04.083027 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:04.083005    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659\": container with ID starting with 0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659 not found: ID does not exist" containerID="0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659"
Sep 11 23:25:04.083073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:04.083034    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659"} err="failed to get container status \"0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659\": rpc error: code = NotFound desc = could not find container \"0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659\": container with ID starting with 0e14f3a4cfd88099e345c9b276a81abcb04126dc3c440b7ffd8ee976897f6659 not found: ID does not exist"
Sep 11 23:25:04.572317 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:04.572284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:05.309505 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:05.309468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:06.017427 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:06.017389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:06.880731 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:06.880699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:07.208714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.208637    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"]
Sep 11 23:25:07.209087 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.209040    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="217793f3-46d7-4840-92bc-5fc40a8a174e" containerName="test-curl-7604-66f6f4"
Sep 11 23:25:07.209087 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.209056    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="217793f3-46d7-4840-92bc-5fc40a8a174e" containerName="test-curl-7604-66f6f4"
Sep 11 23:25:07.209159 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.209131    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="217793f3-46d7-4840-92bc-5fc40a8a174e" containerName="test-curl-7604-66f6f4"
Sep 11 23:25:07.213532 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.213512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.216497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.216477    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"e2e-conflict-b-8c150bb3-kserve-self-signed-certs\""
Sep 11 23:25:07.223124 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.223103    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"]
Sep 11 23:25:07.236417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.236384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:07.346729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.346693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-tmp-dir\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.346729 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.346735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-model-cache\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.346936 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.346756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-home\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.346936 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.346823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e883a075-58c8-4a11-a0cd-c79df0567564-tls-certs\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.346936 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.346867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-dshm\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.346936 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.346891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kzz8\" (UniqueName: \"kubernetes.io/projected/e883a075-58c8-4a11-a0cd-c79df0567564-kube-api-access-7kzz8\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.447705 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.447672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-dshm\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.447851 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.447715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kzz8\" (UniqueName: \"kubernetes.io/projected/e883a075-58c8-4a11-a0cd-c79df0567564-kube-api-access-7kzz8\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.447851 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.447771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-tmp-dir\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.447851 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.447797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-model-cache\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.447851 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.447824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-home\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.448122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.447886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e883a075-58c8-4a11-a0cd-c79df0567564-tls-certs\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.448191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.448167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-model-cache\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.448254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.448191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-tmp-dir\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.448314 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.448248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-home\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.449962 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.449931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-dshm\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.450283 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.450263    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e883a075-58c8-4a11-a0cd-c79df0567564-tls-certs\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.455923 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.455896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kzz8\" (UniqueName: \"kubernetes.io/projected/e883a075-58c8-4a11-a0cd-c79df0567564-kube-api-access-7kzz8\") pod \"e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") " pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.523666 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.523610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:07.655345 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.655297    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:07.794321 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.794211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"]
Sep 11 23:25:07.796944 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:25:07.796887    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode883a075_58c8_4a11_a0cd_c79df0567564.slice/crio-ccf430d3c5a0b3cf9f53a954b5e062cf61cabfcacd0ea8247b82f2e4c86c6ca3 WatchSource:0}: Error finding container ccf430d3c5a0b3cf9f53a954b5e062cf61cabfcacd0ea8247b82f2e4c86c6ca3: Status 404 returned error can't find the container with id ccf430d3c5a0b3cf9f53a954b5e062cf61cabfcacd0ea8247b82f2e4c86c6ca3
Sep 11 23:25:07.943329 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:07.943290    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w" podUID="8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" containerName="main" containerID="cri-o://c5b05445894d15b397113cd52abba6d80364aa84ec215711e857a190db49edf6" gracePeriod=45
Sep 11 23:25:08.103327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.103285    2580 generic.go:356] "Generic (PLEG): container finished" podID="8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" containerID="c5b05445894d15b397113cd52abba6d80364aa84ec215711e857a190db49edf6" exitCode=0
Sep 11 23:25:08.103490 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.103355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w" event={"ID":"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a","Type":"ContainerDied","Data":"c5b05445894d15b397113cd52abba6d80364aa84ec215711e857a190db49edf6"}
Sep 11 23:25:08.105192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.105159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q" event={"ID":"e883a075-58c8-4a11-a0cd-c79df0567564","Type":"ContainerStarted","Data":"af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5"}
Sep 11 23:25:08.105316 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.105195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q" event={"ID":"e883a075-58c8-4a11-a0cd-c79df0567564","Type":"ContainerStarted","Data":"ccf430d3c5a0b3cf9f53a954b5e062cf61cabfcacd0ea8247b82f2e4c86c6ca3"}
Sep 11 23:25:08.124007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.123956    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q" podStartSLOduration=1.123937425 podStartE2EDuration="1.123937425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:25:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:25:08.122908388 +0000 UTC m=+1423.334879565" watchObservedRunningTime="2026-09-11 23:25:08.123937425 +0000 UTC m=+1423.335908600"
Sep 11 23:25:08.182799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.182778    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:25:08.254768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.254739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-dshm\") pod \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") "
Sep 11 23:25:08.254768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.254772    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tls-certs\") pod \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") "
Sep 11 23:25:08.255278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.254797    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lf5b\" (UniqueName: \"kubernetes.io/projected/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-kube-api-access-9lf5b\") pod \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") "
Sep 11 23:25:08.255278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.254827    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tmp-dir\") pod \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") "
Sep 11 23:25:08.255278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.254879    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-model-cache\") pod \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") "
Sep 11 23:25:08.255278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.254943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-home\") pod \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\" (UID: \"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a\") "
Sep 11 23:25:08.255278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.255174    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" (UID: "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:08.255278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.255188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-model-cache" (OuterVolumeSpecName: "model-cache") pod "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" (UID: "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:08.255278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.255267    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-home" (OuterVolumeSpecName: "home") pod "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" (UID: "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:08.255553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.255343    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:08.255553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.255356    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:08.255553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.255365    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:08.257028 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.257011    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" (UID: "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:25:08.257297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.257273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-kube-api-access-9lf5b" (OuterVolumeSpecName: "kube-api-access-9lf5b") pod "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" (UID: "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a"). InnerVolumeSpecName "kube-api-access-9lf5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:08.257297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.257278    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-dshm" (OuterVolumeSpecName: "dshm") pod "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" (UID: "8145eb83-4d17-4fc1-94ab-8e8b6bf1630a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:08.355971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.355940    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:08.355971 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.355968    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:08.356163 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.355982    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lf5b\" (UniqueName: \"kubernetes.io/projected/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a-kube-api-access-9lf5b\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:08.466090 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.466054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:08.565840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.565802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:08.621377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.621294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:08.963726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:08.963628    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:09.041685 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.041652    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:09.107841 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.107808    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"]
Sep 11 23:25:09.108323 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.108303    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" containerName="main"
Sep 11 23:25:09.108403 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.108324    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" containerName="main"
Sep 11 23:25:09.108403 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.108389    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" containerName="main"
Sep 11 23:25:09.111516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.111494    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"
Sep 11 23:25:09.111668 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.111644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w" event={"ID":"8145eb83-4d17-4fc1-94ab-8e8b6bf1630a","Type":"ContainerDied","Data":"9490fc61f746dba4e1106e2bc1027863f34ab7fe231f4c3fe6ad8ab965727eb7"}
Sep 11 23:25:09.111727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.111684    2580 scope.go:117] "RemoveContainer" containerID="c5b05445894d15b397113cd52abba6d80364aa84ec215711e857a190db49edf6"
Sep 11 23:25:09.111872 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.111852    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.115533 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.115510    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b96fb0e1\""
Sep 11 23:25:09.115649 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.115514    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b96fb0e1-dockercfg-dgqp5\""
Sep 11 23:25:09.124547 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.124526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"]
Sep 11 23:25:09.161375 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.161348    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"]
Sep 11 23:25:09.162552 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.162529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3345f82a-acf2-4595-95af-a7da4f591c99-plugins-config-volume\") pod \"payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") " pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.162686 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.162571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nqcz\" (UniqueName: \"kubernetes.io/projected/3345f82a-acf2-4595-95af-a7da4f591c99-kube-api-access-4nqcz\") pod \"payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") " pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.165846 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.165826    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-92d8ce/test-model-92d8ce-kserve-85fbcd9fcf-wst6w"]
Sep 11 23:25:09.263657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.263583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3345f82a-acf2-4595-95af-a7da4f591c99-plugins-config-volume\") pod \"payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") " pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.263657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.263627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nqcz\" (UniqueName: \"kubernetes.io/projected/3345f82a-acf2-4595-95af-a7da4f591c99-kube-api-access-4nqcz\") pod \"payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") " pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.264540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.264516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3345f82a-acf2-4595-95af-a7da4f591c99-plugins-config-volume\") pod \"payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") " pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.273160 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.273130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nqcz\" (UniqueName: \"kubernetes.io/projected/3345f82a-acf2-4595-95af-a7da4f591c99-kube-api-access-4nqcz\") pod \"payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") " pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.299957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.299925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:09.336743 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.336712    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"]
Sep 11 23:25:09.337009 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.336980    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8" podUID="e8c56da4-71e6-4785-a628-257c5b02f4b4" containerName="payload-processing" containerID="cri-o://5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5" gracePeriod=30
Sep 11 23:25:09.347582 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.347556    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"]
Sep 11 23:25:09.347766 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.347745    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb" podUID="f056b4ef-c290-4143-936b-d675cc2fd91b" containerName="payload-pre-processing" containerID="cri-o://cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25" gracePeriod=30
Sep 11 23:25:09.388567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.388540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:09.415795 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.415767    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8145eb83-4d17-4fc1-94ab-8e8b6bf1630a" path="/var/lib/kubelet/pods/8145eb83-4d17-4fc1-94ab-8e8b6bf1630a/volumes"
Sep 11 23:25:09.425740 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.425716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:09.453368 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.453317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:09.555417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.555383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:09.639036 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.639013    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:25:09.641891 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.641872    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:25:09.653494 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.653468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:09.698980 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.698849    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"]
Sep 11 23:25:09.701631 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:25:09.701608    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3345f82a_acf2_4595_95af_a7da4f591c99.slice/crio-2733db3bbc733427519f03cb454d7c0f1d9e8f7e445b53ee9a6d6e181231b24a WatchSource:0}: Error finding container 2733db3bbc733427519f03cb454d7c0f1d9e8f7e445b53ee9a6d6e181231b24a: Status 404 returned error can't find the container with id 2733db3bbc733427519f03cb454d7c0f1d9e8f7e445b53ee9a6d6e181231b24a
Sep 11 23:25:09.770082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.770056    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f056b4ef-c290-4143-936b-d675cc2fd91b-plugins-config-volume\") pod \"f056b4ef-c290-4143-936b-d675cc2fd91b\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") "
Sep 11 23:25:09.770199 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.770173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4dzk\" (UniqueName: \"kubernetes.io/projected/f056b4ef-c290-4143-936b-d675cc2fd91b-kube-api-access-c4dzk\") pod \"f056b4ef-c290-4143-936b-d675cc2fd91b\" (UID: \"f056b4ef-c290-4143-936b-d675cc2fd91b\") "
Sep 11 23:25:09.770272 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.770199    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcrvn\" (UniqueName: \"kubernetes.io/projected/e8c56da4-71e6-4785-a628-257c5b02f4b4-kube-api-access-qcrvn\") pod \"e8c56da4-71e6-4785-a628-257c5b02f4b4\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") "
Sep 11 23:25:09.770272 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.770256    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c56da4-71e6-4785-a628-257c5b02f4b4-plugins-config-volume\") pod \"e8c56da4-71e6-4785-a628-257c5b02f4b4\" (UID: \"e8c56da4-71e6-4785-a628-257c5b02f4b4\") "
Sep 11 23:25:09.770382 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.770350    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f056b4ef-c290-4143-936b-d675cc2fd91b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f056b4ef-c290-4143-936b-d675cc2fd91b" (UID: "f056b4ef-c290-4143-936b-d675cc2fd91b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:25:09.770542 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.770521    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f056b4ef-c290-4143-936b-d675cc2fd91b-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.770636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.770617    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c56da4-71e6-4785-a628-257c5b02f4b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e8c56da4-71e6-4785-a628-257c5b02f4b4" (UID: "e8c56da4-71e6-4785-a628-257c5b02f4b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:25:09.772322 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.772268    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c56da4-71e6-4785-a628-257c5b02f4b4-kube-api-access-qcrvn" (OuterVolumeSpecName: "kube-api-access-qcrvn") pod "e8c56da4-71e6-4785-a628-257c5b02f4b4" (UID: "e8c56da4-71e6-4785-a628-257c5b02f4b4"). InnerVolumeSpecName "kube-api-access-qcrvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:09.772322 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.772275    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f056b4ef-c290-4143-936b-d675cc2fd91b-kube-api-access-c4dzk" (OuterVolumeSpecName: "kube-api-access-c4dzk") pod "f056b4ef-c290-4143-936b-d675cc2fd91b" (UID: "f056b4ef-c290-4143-936b-d675cc2fd91b"). InnerVolumeSpecName "kube-api-access-c4dzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:09.871269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.871240    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4dzk\" (UniqueName: \"kubernetes.io/projected/f056b4ef-c290-4143-936b-d675cc2fd91b-kube-api-access-c4dzk\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.871269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.871268    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcrvn\" (UniqueName: \"kubernetes.io/projected/e8c56da4-71e6-4785-a628-257c5b02f4b4-kube-api-access-qcrvn\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.871488 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.871278    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c56da4-71e6-4785-a628-257c5b02f4b4-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.871968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:09.871939    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:10.120630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.120591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2" event={"ID":"3345f82a-acf2-4595-95af-a7da4f591c99","Type":"ContainerStarted","Data":"e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb"}
Sep 11 23:25:10.120815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.120661    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2" event={"ID":"3345f82a-acf2-4595-95af-a7da4f591c99","Type":"ContainerStarted","Data":"2733db3bbc733427519f03cb454d7c0f1d9e8f7e445b53ee9a6d6e181231b24a"}
Sep 11 23:25:10.120815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.120717    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:10.121925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.121888    2580 generic.go:356] "Generic (PLEG): container finished" podID="e8c56da4-71e6-4785-a628-257c5b02f4b4" containerID="5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5" exitCode=0
Sep 11 23:25:10.122028 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.121941    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"
Sep 11 23:25:10.122028 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.121968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8" event={"ID":"e8c56da4-71e6-4785-a628-257c5b02f4b4","Type":"ContainerDied","Data":"5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5"}
Sep 11 23:25:10.122028 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.122009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8" event={"ID":"e8c56da4-71e6-4785-a628-257c5b02f4b4","Type":"ContainerDied","Data":"432f6fbcf020e36a589d042b7b87e97622738a479a9fa9ce4f27a5daf03678c7"}
Sep 11 23:25:10.122191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.122031    2580 scope.go:117] "RemoveContainer" containerID="5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5"
Sep 11 23:25:10.123416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.123393    2580 generic.go:356] "Generic (PLEG): container finished" podID="f056b4ef-c290-4143-936b-d675cc2fd91b" containerID="cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25" exitCode=0
Sep 11 23:25:10.123493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.123440    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"
Sep 11 23:25:10.123493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.123477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb" event={"ID":"f056b4ef-c290-4143-936b-d675cc2fd91b","Type":"ContainerDied","Data":"cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25"}
Sep 11 23:25:10.123595 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.123523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb" event={"ID":"f056b4ef-c290-4143-936b-d675cc2fd91b","Type":"ContainerDied","Data":"26aa5deee1b8decacc20b19727fec574aab1138c5daa0abbf1f9d80f9de918f1"}
Sep 11 23:25:10.131831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.131815    2580 scope.go:117] "RemoveContainer" containerID="5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5"
Sep 11 23:25:10.132138 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:10.132121    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5\": container with ID starting with 5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5 not found: ID does not exist" containerID="5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5"
Sep 11 23:25:10.132220 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.132146    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5"} err="failed to get container status \"5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5\": rpc error: code = NotFound desc = could not find container \"5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5\": container with ID starting with 5284bceb65d3e8c540feb5be6945a439ab40bfd571e5a5e2985932d6bf9947a5 not found: ID does not exist"
Sep 11 23:25:10.132220 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.132163    2580 scope.go:117] "RemoveContainer" containerID="cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25"
Sep 11 23:25:10.138756 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.138720    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2" podStartSLOduration=1.138706101 podStartE2EDuration="1.138706101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:25:10.137770679 +0000 UTC m=+1425.349741853" watchObservedRunningTime="2026-09-11 23:25:10.138706101 +0000 UTC m=+1425.350677274"
Sep 11 23:25:10.140191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.140173    2580 scope.go:117] "RemoveContainer" containerID="cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25"
Sep 11 23:25:10.140447 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:10.140427    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25\": container with ID starting with cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25 not found: ID does not exist" containerID="cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25"
Sep 11 23:25:10.140500 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.140454    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25"} err="failed to get container status \"cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25\": rpc error: code = NotFound desc = could not find container \"cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25\": container with ID starting with cd795196db5674c979669b6fff05d5d219f36bd1ca51c732db4a478679da5b25 not found: ID does not exist"
Sep 11 23:25:10.151589 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.151566    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"]
Sep 11 23:25:10.153767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.153745    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-92d8ce-6db769d994-xsnnb"]
Sep 11 23:25:10.167493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.167472    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"]
Sep 11 23:25:10.171093 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.171071    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-92d8ce-58865c97ff-vl8t8"]
Sep 11 23:25:10.250174 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:10.250142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:11.160244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:11.160202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:11.414463 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:11.414384    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c56da4-71e6-4785-a628-257c5b02f4b4" path="/var/lib/kubelet/pods/e8c56da4-71e6-4785-a628-257c5b02f4b4/volumes"
Sep 11 23:25:11.414753 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:11.414741    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f056b4ef-c290-4143-936b-d675cc2fd91b" path="/var/lib/kubelet/pods/f056b4ef-c290-4143-936b-d675cc2fd91b/volumes"
Sep 11 23:25:12.164348 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:12.164313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:12.914820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:12.914780    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:13.866892 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:13.866847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:14.558857 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:14.558817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:15.282431 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:15.282396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:16.054468 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:16.054438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:16.115859 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:16.115822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:16.310360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:16.310275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:16.876353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:16.876319    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:17.523824 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:17.523785    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:17.524231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:17.523837    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:17.536156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:17.536129    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:17.745693 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:17.745660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:18.177354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:18.177327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:25:18.551759 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:18.551669    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:19.271017 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:19.270966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:20.236657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:20.236618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:21.027507 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:21.027472    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:21.132776 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:21.132750    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:25:21.951654 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:21.951619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:22.851556 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:22.851521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:23.128433 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:23.128350    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:23.594319 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:23.594288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:24.300466 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:24.300436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:24.398463 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:24.398435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:24.518885 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:24.518849    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:24.635062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:24.635035    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:25.371651 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:25.371622    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:26.085088 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:26.085052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:26.698592 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:26.698556    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"]
Sep 11 23:25:26.699089 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:26.698811    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t" podUID="468f8828-ee8f-4744-8ff8-b8385ac45ffb" containerName="payload-processing" containerID="cri-o://33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a" gracePeriod=30
Sep 11 23:25:26.946953 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:26.946905    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:25:27.043630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.043560    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468f8828-ee8f-4744-8ff8-b8385ac45ffb-plugins-config-volume\") pod \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") "
Sep 11 23:25:27.043630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.043593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxj6c\" (UniqueName: \"kubernetes.io/projected/468f8828-ee8f-4744-8ff8-b8385ac45ffb-kube-api-access-cxj6c\") pod \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\" (UID: \"468f8828-ee8f-4744-8ff8-b8385ac45ffb\") "
Sep 11 23:25:27.043994 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.043973    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468f8828-ee8f-4744-8ff8-b8385ac45ffb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "468f8828-ee8f-4744-8ff8-b8385ac45ffb" (UID: "468f8828-ee8f-4744-8ff8-b8385ac45ffb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:25:27.045678 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.045651    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468f8828-ee8f-4744-8ff8-b8385ac45ffb-kube-api-access-cxj6c" (OuterVolumeSpecName: "kube-api-access-cxj6c") pod "468f8828-ee8f-4744-8ff8-b8385ac45ffb" (UID: "468f8828-ee8f-4744-8ff8-b8385ac45ffb"). InnerVolumeSpecName "kube-api-access-cxj6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:27.144286 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.144259    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468f8828-ee8f-4744-8ff8-b8385ac45ffb-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:27.144286 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.144285    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxj6c\" (UniqueName: \"kubernetes.io/projected/468f8828-ee8f-4744-8ff8-b8385ac45ffb-kube-api-access-cxj6c\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:27.211152 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.211123    2580 generic.go:356] "Generic (PLEG): container finished" podID="468f8828-ee8f-4744-8ff8-b8385ac45ffb" containerID="33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a" exitCode=0
Sep 11 23:25:27.211260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.211173    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"
Sep 11 23:25:27.211260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.211188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t" event={"ID":"468f8828-ee8f-4744-8ff8-b8385ac45ffb","Type":"ContainerDied","Data":"33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a"}
Sep 11 23:25:27.211260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.211230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t" event={"ID":"468f8828-ee8f-4744-8ff8-b8385ac45ffb","Type":"ContainerDied","Data":"91a5aed04f507a94bff3a42aef73c076d33ad9e1c684f22c7bf69bce25c58652"}
Sep 11 23:25:27.211260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.211253    2580 scope.go:117] "RemoveContainer" containerID="33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a"
Sep 11 23:25:27.220479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.220461    2580 scope.go:117] "RemoveContainer" containerID="33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a"
Sep 11 23:25:27.220756 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:27.220738    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a\": container with ID starting with 33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a not found: ID does not exist" containerID="33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a"
Sep 11 23:25:27.220803 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.220765    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a"} err="failed to get container status \"33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a\": rpc error: code = NotFound desc = could not find container \"33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a\": container with ID starting with 33032f0f59612500ba92af92111328a7c9132e75e612b8290625b9814619ba6a not found: ID does not exist"
Sep 11 23:25:27.250717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.250695    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"]
Sep 11 23:25:27.262361 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.262337    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bfa694-6948794495-k9f5t"]
Sep 11 23:25:27.413518 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.413490    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468f8828-ee8f-4744-8ff8-b8385ac45ffb" path="/var/lib/kubelet/pods/468f8828-ee8f-4744-8ff8-b8385ac45ffb/volumes"
Sep 11 23:25:27.548714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.548690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:27.649314 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.649285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:27.847885 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:27.847856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:28.616766 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:28.616726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:29.353139 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:29.353105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:29.643718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:29.643641    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"]
Sep 11 23:25:29.726466 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:29.726431    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:30.158158 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:30.158117    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:30.848979 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:30.848944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:33.070874 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:33.070833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:33.141203 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:33.141173    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:33.325332 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:33.325247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:33.876394 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:33.876367    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:34.232721 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:34.232644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:34.464933 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:34.464885    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:36.956376 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:36.956342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:37.623968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:37.623906    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:37.716002 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:37.715968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:37.812898 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:37.812864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:38.590449 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:38.590421    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:39.329454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:39.329420    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:40.163621 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:40.163588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:40.883570 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:40.883527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:41.649983 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:41.649944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:41.754544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:41.754510    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:41.854727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:41.854692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:42.147787 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.147756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:42.220377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.220343    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:42.257014 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.256982    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"]
Sep 11 23:25:42.257369 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257342    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f056b4ef-c290-4143-936b-d675cc2fd91b" containerName="payload-pre-processing"
Sep 11 23:25:42.257369 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257363    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f056b4ef-c290-4143-936b-d675cc2fd91b" containerName="payload-pre-processing"
Sep 11 23:25:42.257540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257374    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c56da4-71e6-4785-a628-257c5b02f4b4" containerName="payload-processing"
Sep 11 23:25:42.257540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257383    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c56da4-71e6-4785-a628-257c5b02f4b4" containerName="payload-processing"
Sep 11 23:25:42.257540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257399    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468f8828-ee8f-4744-8ff8-b8385ac45ffb" containerName="payload-processing"
Sep 11 23:25:42.257540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257408    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="468f8828-ee8f-4744-8ff8-b8385ac45ffb" containerName="payload-processing"
Sep 11 23:25:42.257540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257519    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c56da4-71e6-4785-a628-257c5b02f4b4" containerName="payload-processing"
Sep 11 23:25:42.257540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257535    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f056b4ef-c290-4143-936b-d675cc2fd91b" containerName="payload-pre-processing"
Sep 11 23:25:42.257811 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.257547    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="468f8828-ee8f-4744-8ff8-b8385ac45ffb" containerName="payload-processing"
Sep 11 23:25:42.261845 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.261828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.264618 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.264601    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-1c4628\""
Sep 11 23:25:42.270758 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.270731    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"]
Sep 11 23:25:42.293309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.293284    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"]
Sep 11 23:25:42.296615 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.296600    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.299553 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.299537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-1c4628-dockercfg-gf8rl\""
Sep 11 23:25:42.304996 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.304969    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"]
Sep 11 23:25:42.381327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.381296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1519560-2009-43ad-ba17-59b74e4cb7cd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.381504 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.381344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1da8b65-21d5-42c8-826b-3a3f31169755-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.381504 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.381475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2s8z\" (UniqueName: \"kubernetes.io/projected/d1da8b65-21d5-42c8-826b-3a3f31169755-kube-api-access-x2s8z\") pod \"payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.381614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.381514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fkf7\" (UniqueName: \"kubernetes.io/projected/c1519560-2009-43ad-ba17-59b74e4cb7cd-kube-api-access-4fkf7\") pod \"payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.482984 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.482876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1519560-2009-43ad-ba17-59b74e4cb7cd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.482984 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.482970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1da8b65-21d5-42c8-826b-3a3f31169755-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.483169 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.483032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2s8z\" (UniqueName: \"kubernetes.io/projected/d1da8b65-21d5-42c8-826b-3a3f31169755-kube-api-access-x2s8z\") pod \"payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.483169 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.483142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fkf7\" (UniqueName: \"kubernetes.io/projected/c1519560-2009-43ad-ba17-59b74e4cb7cd-kube-api-access-4fkf7\") pod \"payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.483599 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.483579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1519560-2009-43ad-ba17-59b74e4cb7cd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.483683 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.483628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1da8b65-21d5-42c8-826b-3a3f31169755-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.500666 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.500640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2s8z\" (UniqueName: \"kubernetes.io/projected/d1da8b65-21d5-42c8-826b-3a3f31169755-kube-api-access-x2s8z\") pod \"payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.504577 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.504551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fkf7\" (UniqueName: \"kubernetes.io/projected/c1519560-2009-43ad-ba17-59b74e4cb7cd-kube-api-access-4fkf7\") pod \"payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.572572 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.572538    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:42.607945 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.607899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:42.849242 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.849218    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"]
Sep 11 23:25:42.851072 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:25:42.851038    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1da8b65_21d5_42c8_826b_3a3f31169755.slice/crio-e74598e40bfc95005d2c09b0c4c00cf5a6c446a90e27851dde894bd2b5d8a579 WatchSource:0}: Error finding container e74598e40bfc95005d2c09b0c4c00cf5a6c446a90e27851dde894bd2b5d8a579: Status 404 returned error can't find the container with id e74598e40bfc95005d2c09b0c4c00cf5a6c446a90e27851dde894bd2b5d8a579
Sep 11 23:25:42.890719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:42.890698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"]
Sep 11 23:25:42.892996 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:25:42.892969    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1519560_2009_43ad_ba17_59b74e4cb7cd.slice/crio-34945b57674a61d5533b5f03d0b1f893ce0e59ae7e1f0abada6502ae730cf5e2 WatchSource:0}: Error finding container 34945b57674a61d5533b5f03d0b1f893ce0e59ae7e1f0abada6502ae730cf5e2: Status 404 returned error can't find the container with id 34945b57674a61d5533b5f03d0b1f893ce0e59ae7e1f0abada6502ae730cf5e2
Sep 11 23:25:43.287411 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.287310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px" event={"ID":"c1519560-2009-43ad-ba17-59b74e4cb7cd","Type":"ContainerStarted","Data":"75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592"}
Sep 11 23:25:43.287556 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.287473    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:25:43.287556 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.287512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px" event={"ID":"c1519560-2009-43ad-ba17-59b74e4cb7cd","Type":"ContainerStarted","Data":"34945b57674a61d5533b5f03d0b1f893ce0e59ae7e1f0abada6502ae730cf5e2"}
Sep 11 23:25:43.288954 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.288926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w" event={"ID":"d1da8b65-21d5-42c8-826b-3a3f31169755","Type":"ContainerStarted","Data":"ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85"}
Sep 11 23:25:43.289077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.288970    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:43.289077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.288987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w" event={"ID":"d1da8b65-21d5-42c8-826b-3a3f31169755","Type":"ContainerStarted","Data":"e74598e40bfc95005d2c09b0c4c00cf5a6c446a90e27851dde894bd2b5d8a579"}
Sep 11 23:25:43.305867 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.305826    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px" podStartSLOduration=1.305814302 podStartE2EDuration="1.305814302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:25:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:25:43.3037335 +0000 UTC m=+1458.515704677" watchObservedRunningTime="2026-09-11 23:25:43.305814302 +0000 UTC m=+1458.517785500"
Sep 11 23:25:43.320045 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:43.320006    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w" podStartSLOduration=1.319994536 podStartE2EDuration="1.319994536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:25:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:25:43.319002128 +0000 UTC m=+1458.530973303" watchObservedRunningTime="2026-09-11 23:25:43.319994536 +0000 UTC m=+1458.531965711"
Sep 11 23:25:44.662578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:44.662539    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd" podUID="84599859-20e2-4aca-9398-35671efbef7d" containerName="main" containerID="cri-o://1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444" gracePeriod=45
Sep 11 23:25:44.921670 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:44.921609    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:25:45.005397 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005358    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84599859-20e2-4aca-9398-35671efbef7d-tls-certs\") pod \"84599859-20e2-4aca-9398-35671efbef7d\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") "
Sep 11 23:25:45.005565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005408    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-home\") pod \"84599859-20e2-4aca-9398-35671efbef7d\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") "
Sep 11 23:25:45.005565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005442    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-tmp-dir\") pod \"84599859-20e2-4aca-9398-35671efbef7d\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") "
Sep 11 23:25:45.005565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c4dw\" (UniqueName: \"kubernetes.io/projected/84599859-20e2-4aca-9398-35671efbef7d-kube-api-access-6c4dw\") pod \"84599859-20e2-4aca-9398-35671efbef7d\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") "
Sep 11 23:25:45.005565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005527    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-model-cache\") pod \"84599859-20e2-4aca-9398-35671efbef7d\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") "
Sep 11 23:25:45.005565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-dshm\") pod \"84599859-20e2-4aca-9398-35671efbef7d\" (UID: \"84599859-20e2-4aca-9398-35671efbef7d\") "
Sep 11 23:25:45.005840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005684    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-home" (OuterVolumeSpecName: "home") pod "84599859-20e2-4aca-9398-35671efbef7d" (UID: "84599859-20e2-4aca-9398-35671efbef7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:45.005840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005729    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84599859-20e2-4aca-9398-35671efbef7d" (UID: "84599859-20e2-4aca-9398-35671efbef7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:45.005840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005804    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "84599859-20e2-4aca-9398-35671efbef7d" (UID: "84599859-20e2-4aca-9398-35671efbef7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:45.006003 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005973    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:45.006003 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.005996    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:45.006107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.006009    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:45.007547 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.007519    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-dshm" (OuterVolumeSpecName: "dshm") pod "84599859-20e2-4aca-9398-35671efbef7d" (UID: "84599859-20e2-4aca-9398-35671efbef7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:25:45.007648 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.007594    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84599859-20e2-4aca-9398-35671efbef7d-kube-api-access-6c4dw" (OuterVolumeSpecName: "kube-api-access-6c4dw") pod "84599859-20e2-4aca-9398-35671efbef7d" (UID: "84599859-20e2-4aca-9398-35671efbef7d"). InnerVolumeSpecName "kube-api-access-6c4dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:45.007705 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.007652    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84599859-20e2-4aca-9398-35671efbef7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84599859-20e2-4aca-9398-35671efbef7d" (UID: "84599859-20e2-4aca-9398-35671efbef7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:25:45.106696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.106644    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84599859-20e2-4aca-9398-35671efbef7d-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:45.106696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.106688    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c4dw\" (UniqueName: \"kubernetes.io/projected/84599859-20e2-4aca-9398-35671efbef7d-kube-api-access-6c4dw\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:45.106696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.106700    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84599859-20e2-4aca-9398-35671efbef7d-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:25:45.299714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.299625    2580 generic.go:356] "Generic (PLEG): container finished" podID="84599859-20e2-4aca-9398-35671efbef7d" containerID="1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444" exitCode=0
Sep 11 23:25:45.299714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.299668    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"
Sep 11 23:25:45.299943 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.299712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd" event={"ID":"84599859-20e2-4aca-9398-35671efbef7d","Type":"ContainerDied","Data":"1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444"}
Sep 11 23:25:45.299943 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.299751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd" event={"ID":"84599859-20e2-4aca-9398-35671efbef7d","Type":"ContainerDied","Data":"9b56cae4d5d8cc6ec7978f6c352a3f3b09c21694b0a73a5d8eab19c6a3c114b0"}
Sep 11 23:25:45.299943 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.299769    2580 scope.go:117] "RemoveContainer" containerID="1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444"
Sep 11 23:25:45.316162 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.316143    2580 scope.go:117] "RemoveContainer" containerID="1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444"
Sep 11 23:25:45.316424 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:25:45.316404    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444\": container with ID starting with 1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444 not found: ID does not exist" containerID="1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444"
Sep 11 23:25:45.316493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.316431    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444"} err="failed to get container status \"1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444\": rpc error: code = NotFound desc = could not find container \"1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444\": container with ID starting with 1a07568ef48a9431ccba15a71d78122165b5dede072b6d7f63d8a807ba6f5444 not found: ID does not exist"
Sep 11 23:25:45.326888 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.326861    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"]
Sep 11 23:25:45.330725 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.330705    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-c0d4bf/noten-model-c0d4bf-kserve-6bf6f89699-hmdwd"]
Sep 11 23:25:45.415149 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:45.415120    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84599859-20e2-4aca-9398-35671efbef7d" path="/var/lib/kubelet/pods/84599859-20e2-4aca-9398-35671efbef7d/volumes"
Sep 11 23:25:47.246876 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:47.246841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:25:54.295980 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:54.295947    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:25:54.296459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:25:54.296026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:26:00.533502 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:00.533460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:02.597029 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:02.596997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:03.622931 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:03.622890    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:05.144270 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:05.144231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:05.513972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:05.513883    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:06.010673 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:06.010636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:06.635121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:06.635084    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:06.844747 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:06.844713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:07.762197 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:07.762151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:07.862856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:07.862822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:08.028486 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:08.028396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:08.338411 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:08.338375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:08.663472 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:08.663386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:08.765467 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:08.765436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:09.424928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:09.424878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:10.248373 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:10.248332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:11.276952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:11.276889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:11.942436 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:11.942404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:12.017977 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:12.017943    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:12.095021 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:12.094977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:13.152925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:13.152869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:13.885192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:13.885155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:14.602962 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:14.602908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:15.466743 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:15.466707    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:16.163439 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:16.163410    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:16.945260 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:16.945212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:17.047290 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.047260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:17.168423 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.168389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:17.542815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.542781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:17.642409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.642374    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:17.644010 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.643986    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"]
Sep 11 23:26:17.644504 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.644486    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84599859-20e2-4aca-9398-35671efbef7d" containerName="main"
Sep 11 23:26:17.644573 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.644506    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="84599859-20e2-4aca-9398-35671efbef7d" containerName="main"
Sep 11 23:26:17.644637 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.644625    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="84599859-20e2-4aca-9398-35671efbef7d" containerName="main"
Sep 11 23:26:17.647809 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.647794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.650697 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.650681    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-da9451\""
Sep 11 23:26:17.655767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.655746    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"]
Sep 11 23:26:17.659244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.659226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:17.659840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.659822    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"]
Sep 11 23:26:17.662055 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.662034    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-da9451-dockercfg-pq5mb\""
Sep 11 23:26:17.670513 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.670493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"]
Sep 11 23:26:17.699121 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.699089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:17.699268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.699125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr7xb\" (UniqueName: \"kubernetes.io/projected/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-kube-api-access-cr7xb\") pod \"payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:17.699268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.699160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4d67a6-d434-4bd6-88d4-581181f3e751-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.699389 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.699315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlxx9\" (UniqueName: \"kubernetes.io/projected/1a4d67a6-d434-4bd6-88d4-581181f3e751-kube-api-access-mlxx9\") pod \"payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.800452 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.800379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:17.800452 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.800413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr7xb\" (UniqueName: \"kubernetes.io/projected/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-kube-api-access-cr7xb\") pod \"payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:17.800452 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.800449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4d67a6-d434-4bd6-88d4-581181f3e751-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.800667 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.800510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlxx9\" (UniqueName: \"kubernetes.io/projected/1a4d67a6-d434-4bd6-88d4-581181f3e751-kube-api-access-mlxx9\") pod \"payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.801170 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.801150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:17.801249 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.801233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4d67a6-d434-4bd6-88d4-581181f3e751-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.809009 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.808982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlxx9\" (UniqueName: \"kubernetes.io/projected/1a4d67a6-d434-4bd6-88d4-581181f3e751-kube-api-access-mlxx9\") pod \"payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.809122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.809028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr7xb\" (UniqueName: \"kubernetes.io/projected/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-kube-api-access-cr7xb\") pod \"payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:17.958413 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.958383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:17.970105 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:17.970077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:18.232478 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.232448    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"]
Sep 11 23:26:18.235720 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:26:18.235695    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4d67a6_d434_4bd6_88d4_581181f3e751.slice/crio-15bbf869015339648c02b2f837723cada3ee9f24ef4c1ba857b49cdba2a680db WatchSource:0}: Error finding container 15bbf869015339648c02b2f837723cada3ee9f24ef4c1ba857b49cdba2a680db: Status 404 returned error can't find the container with id 15bbf869015339648c02b2f837723cada3ee9f24ef4c1ba857b49cdba2a680db
Sep 11 23:26:18.261797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.261771    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"]
Sep 11 23:26:18.267108 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:26:18.267079    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2519f6bf_5488_4cbb_8f84_1afbd95af1d3.slice/crio-8b4bfe4d9f1bfadad76dc66d518edebc83dd0582f9476ad1233609a28b122a67 WatchSource:0}: Error finding container 8b4bfe4d9f1bfadad76dc66d518edebc83dd0582f9476ad1233609a28b122a67: Status 404 returned error can't find the container with id 8b4bfe4d9f1bfadad76dc66d518edebc83dd0582f9476ad1233609a28b122a67
Sep 11 23:26:18.458461 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.458414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9" event={"ID":"1a4d67a6-d434-4bd6-88d4-581181f3e751","Type":"ContainerStarted","Data":"935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110"}
Sep 11 23:26:18.458649 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.458482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9" event={"ID":"1a4d67a6-d434-4bd6-88d4-581181f3e751","Type":"ContainerStarted","Data":"15bbf869015339648c02b2f837723cada3ee9f24ef4c1ba857b49cdba2a680db"}
Sep 11 23:26:18.458649 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.458511    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:18.460430 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.460345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc" event={"ID":"2519f6bf-5488-4cbb-8f84-1afbd95af1d3","Type":"ContainerStarted","Data":"83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6"}
Sep 11 23:26:18.460430 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.460395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc" event={"ID":"2519f6bf-5488-4cbb-8f84-1afbd95af1d3","Type":"ContainerStarted","Data":"8b4bfe4d9f1bfadad76dc66d518edebc83dd0582f9476ad1233609a28b122a67"}
Sep 11 23:26:18.460658 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.460475    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:18.475605 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.475562    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9" podStartSLOduration=1.475548855 podStartE2EDuration="1.475548855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:26:18.474849527 +0000 UTC m=+1493.686820704" watchObservedRunningTime="2026-09-11 23:26:18.475548855 +0000 UTC m=+1493.687520100"
Sep 11 23:26:18.493957 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:18.493859    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc" podStartSLOduration=1.493845425 podStartE2EDuration="1.493845425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:26:18.493257126 +0000 UTC m=+1493.705228300" watchObservedRunningTime="2026-09-11 23:26:18.493845425 +0000 UTC m=+1493.705816598"
Sep 11 23:26:29.467372 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:29.467342    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:26:29.469744 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:29.467435    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:26:39.160631 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:39.160596    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:40.278208 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:40.278142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:40.849642 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:40.849589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:41.564049 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:41.564015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:41.884626 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:41.884539    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:42.465064 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:42.465027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:43.288934 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:43.288881    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:43.976433 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:43.976396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:44.729842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:44.729812    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:49.149298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:49.149214    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"]
Sep 11 23:26:49.173647 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:49.173616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:26:49.870277 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:26:49.870234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:04.168145 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.168079    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q" podUID="e883a075-58c8-4a11-a0cd-c79df0567564" containerName="main" containerID="cri-o://af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5" gracePeriod=45
Sep 11 23:27:04.415165 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.415141    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:27:04.503659 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.503577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kzz8\" (UniqueName: \"kubernetes.io/projected/e883a075-58c8-4a11-a0cd-c79df0567564-kube-api-access-7kzz8\") pod \"e883a075-58c8-4a11-a0cd-c79df0567564\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") "
Sep 11 23:27:04.503659 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.503653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-tmp-dir\") pod \"e883a075-58c8-4a11-a0cd-c79df0567564\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") "
Sep 11 23:27:04.503887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.503678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-model-cache\") pod \"e883a075-58c8-4a11-a0cd-c79df0567564\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") "
Sep 11 23:27:04.503887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.503711    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-home\") pod \"e883a075-58c8-4a11-a0cd-c79df0567564\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") "
Sep 11 23:27:04.503887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.503732    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e883a075-58c8-4a11-a0cd-c79df0567564-tls-certs\") pod \"e883a075-58c8-4a11-a0cd-c79df0567564\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") "
Sep 11 23:27:04.503887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.503756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-dshm\") pod \"e883a075-58c8-4a11-a0cd-c79df0567564\" (UID: \"e883a075-58c8-4a11-a0cd-c79df0567564\") "
Sep 11 23:27:04.504193 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.504163    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-model-cache" (OuterVolumeSpecName: "model-cache") pod "e883a075-58c8-4a11-a0cd-c79df0567564" (UID: "e883a075-58c8-4a11-a0cd-c79df0567564"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:04.504287 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.504188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e883a075-58c8-4a11-a0cd-c79df0567564" (UID: "e883a075-58c8-4a11-a0cd-c79df0567564"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:04.504287 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.504205    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-home" (OuterVolumeSpecName: "home") pod "e883a075-58c8-4a11-a0cd-c79df0567564" (UID: "e883a075-58c8-4a11-a0cd-c79df0567564"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:04.505955 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.505908    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-dshm" (OuterVolumeSpecName: "dshm") pod "e883a075-58c8-4a11-a0cd-c79df0567564" (UID: "e883a075-58c8-4a11-a0cd-c79df0567564"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:04.506077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.505989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e883a075-58c8-4a11-a0cd-c79df0567564-kube-api-access-7kzz8" (OuterVolumeSpecName: "kube-api-access-7kzz8") pod "e883a075-58c8-4a11-a0cd-c79df0567564" (UID: "e883a075-58c8-4a11-a0cd-c79df0567564"). InnerVolumeSpecName "kube-api-access-7kzz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:27:04.506077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.506038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e883a075-58c8-4a11-a0cd-c79df0567564-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e883a075-58c8-4a11-a0cd-c79df0567564" (UID: "e883a075-58c8-4a11-a0cd-c79df0567564"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:27:04.604622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.604574    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:04.604622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.604615    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e883a075-58c8-4a11-a0cd-c79df0567564-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:04.604622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.604624    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:04.604853 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.604633    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kzz8\" (UniqueName: \"kubernetes.io/projected/e883a075-58c8-4a11-a0cd-c79df0567564-kube-api-access-7kzz8\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:04.604853 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.604641    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:04.604853 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.604656    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e883a075-58c8-4a11-a0cd-c79df0567564-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:04.681157 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.681123    2580 generic.go:356] "Generic (PLEG): container finished" podID="e883a075-58c8-4a11-a0cd-c79df0567564" containerID="af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5" exitCode=0
Sep 11 23:27:04.681296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.681185    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"
Sep 11 23:27:04.681296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.681215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q" event={"ID":"e883a075-58c8-4a11-a0cd-c79df0567564","Type":"ContainerDied","Data":"af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5"}
Sep 11 23:27:04.681296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.681262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q" event={"ID":"e883a075-58c8-4a11-a0cd-c79df0567564","Type":"ContainerDied","Data":"ccf430d3c5a0b3cf9f53a954b5e062cf61cabfcacd0ea8247b82f2e4c86c6ca3"}
Sep 11 23:27:04.681296 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.681280    2580 scope.go:117] "RemoveContainer" containerID="af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5"
Sep 11 23:27:04.692460 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.692435    2580 scope.go:117] "RemoveContainer" containerID="af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5"
Sep 11 23:27:04.692789 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:27:04.692765    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5\": container with ID starting with af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5 not found: ID does not exist" containerID="af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5"
Sep 11 23:27:04.692881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.692803    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5"} err="failed to get container status \"af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5\": rpc error: code = NotFound desc = could not find container \"af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5\": container with ID starting with af73bf2a64902fac94b3a09705841d0e0c75d42e151fb8c11c32712ca7313ec5 not found: ID does not exist"
Sep 11 23:27:04.719324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.719300    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"]
Sep 11 23:27:04.727527 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:04.727507    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-b-8c150bb3-kserve-9b67bc6fc-5866q"]
Sep 11 23:27:05.414424 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:05.414392    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e883a075-58c8-4a11-a0cd-c79df0567564" path="/var/lib/kubelet/pods/e883a075-58c8-4a11-a0cd-c79df0567564/volumes"
Sep 11 23:27:14.756999 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:14.756963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:17.317437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.317400    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"]
Sep 11 23:27:17.318026 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.318006    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e883a075-58c8-4a11-a0cd-c79df0567564" containerName="main"
Sep 11 23:27:17.318026 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.318027    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e883a075-58c8-4a11-a0cd-c79df0567564" containerName="main"
Sep 11 23:27:17.318181 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.318115    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e883a075-58c8-4a11-a0cd-c79df0567564" containerName="main"
Sep 11 23:27:17.322868 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.322846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.327321 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.327294    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"kube-root-ca.crt\""
Sep 11 23:27:17.327436 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.327312    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"default-dockercfg-7x9sj\""
Sep 11 23:27:17.327436 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.327348    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"auth-test-model-1c4628-kserve-self-signed-certs\""
Sep 11 23:27:17.327568 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.327478    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"openshift-service-ca.crt\""
Sep 11 23:27:17.332534 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.332512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"]
Sep 11 23:27:17.344679 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.344652    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:17.412855 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.412823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-dshm\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.413039 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.412864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tmp-dir\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.413039 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.412964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-home\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.413039 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.412987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tls-certs\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.413039 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.413034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-model-cache\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.413195 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.413085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdf9r\" (UniqueName: \"kubernetes.io/projected/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-kube-api-access-sdf9r\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.514548 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.514520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-home\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.514548 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.514548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tls-certs\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.514757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.514578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-model-cache\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.514757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.514608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdf9r\" (UniqueName: \"kubernetes.io/projected/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-kube-api-access-sdf9r\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.514757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.514648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-dshm\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.514757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.514677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tmp-dir\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.515018 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.514997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-home\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.515078 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.515023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-model-cache\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.515120 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.515103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tmp-dir\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.516897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.516875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-dshm\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.517085 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.517068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tls-certs\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.523171 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.523149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdf9r\" (UniqueName: \"kubernetes.io/projected/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-kube-api-access-sdf9r\") pod \"auth-test-model-1c4628-kserve-744bd586f5-tdp4z\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.632168 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.632073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:17.903855 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:17.903732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"]
Sep 11 23:27:17.906706 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:27:17.906674    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07635a6a_b0af_422f_9eaa_c85c96ebcbdb.slice/crio-57acfe16b917ad51f5f81762f43dac98cfaeeeffca75d38bfe93342256623609 WatchSource:0}: Error finding container 57acfe16b917ad51f5f81762f43dac98cfaeeeffca75d38bfe93342256623609: Status 404 returned error can't find the container with id 57acfe16b917ad51f5f81762f43dac98cfaeeeffca75d38bfe93342256623609
Sep 11 23:27:18.754564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:18.754531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z" event={"ID":"07635a6a-b0af-422f-9eaa-c85c96ebcbdb","Type":"ContainerStarted","Data":"3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1"}
Sep 11 23:27:18.754978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:18.754574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z" event={"ID":"07635a6a-b0af-422f-9eaa-c85c96ebcbdb","Type":"ContainerStarted","Data":"57acfe16b917ad51f5f81762f43dac98cfaeeeffca75d38bfe93342256623609"}
Sep 11 23:27:18.772033 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:18.771983    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z" podStartSLOduration=1.771968589 podStartE2EDuration="1.771968589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:27:18.771210551 +0000 UTC m=+1553.983181751" watchObservedRunningTime="2026-09-11 23:27:18.771968589 +0000 UTC m=+1553.983939762"
Sep 11 23:27:20.448200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:20.448165    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"]
Sep 11 23:27:20.469237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:20.469203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:27.633170 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:27.633134    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:27.633613 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:27.633350    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:27.645856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:27.645833    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:27.808502 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:27.808472    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:27:30.613108 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.613072    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"]
Sep 11 23:27:30.616797 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.616773    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.619606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.619581    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"distinct-w4-06ef79-kserve-self-signed-certs\""
Sep 11 23:27:30.626985 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.626955    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"]
Sep 11 23:27:30.640323 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.640290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:30.731298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.731265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tls-certs\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.731298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.731304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-home\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.731509 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.731323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-model-cache\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.731509 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.731450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/3845c9a0-7ffe-479b-a088-9b60f62f8c66-kube-api-access-4j42k\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.731582 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.731533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tmp-dir\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.731582 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.731566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-dshm\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.832748 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.832713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tls-certs\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.832952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.832758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-home\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.832952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.832775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-model-cache\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.832952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.832837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/3845c9a0-7ffe-479b-a088-9b60f62f8c66-kube-api-access-4j42k\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.832952 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.832944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tmp-dir\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.833187 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.832991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-dshm\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.833248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.833181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-home\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.833248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.833201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-model-cache\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.833346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.833280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tmp-dir\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.835050 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.835023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-dshm\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.835335 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.835319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tls-certs\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.841189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.841169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/3845c9a0-7ffe-479b-a088-9b60f62f8c66-kube-api-access-4j42k\") pod \"distinct-w4-06ef79-kserve-759c67dc4b-8kp2x\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:30.927422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:30.927343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:31.200425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:31.200396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"]
Sep 11 23:27:31.202536 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:27:31.202512    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3845c9a0_7ffe_479b_a088_9b60f62f8c66.slice/crio-445e394b96fadb3e5191ebc7b2fbfc1a0a10d8afd76e3580a901c3ddd4f989ff WatchSource:0}: Error finding container 445e394b96fadb3e5191ebc7b2fbfc1a0a10d8afd76e3580a901c3ddd4f989ff: Status 404 returned error can't find the container with id 445e394b96fadb3e5191ebc7b2fbfc1a0a10d8afd76e3580a901c3ddd4f989ff
Sep 11 23:27:31.818193 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:31.818152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x" event={"ID":"3845c9a0-7ffe-479b-a088-9b60f62f8c66","Type":"ContainerStarted","Data":"3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633"}
Sep 11 23:27:31.818586 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:31.818205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x" event={"ID":"3845c9a0-7ffe-479b-a088-9b60f62f8c66","Type":"ContainerStarted","Data":"445e394b96fadb3e5191ebc7b2fbfc1a0a10d8afd76e3580a901c3ddd4f989ff"}
Sep 11 23:27:31.838263 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:31.838208    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x" podStartSLOduration=1.8381912489999999 podStartE2EDuration="1.838191249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:27:31.836951825 +0000 UTC m=+1567.048923000" watchObservedRunningTime="2026-09-11 23:27:31.838191249 +0000 UTC m=+1567.050162472"
Sep 11 23:27:35.467199 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.467162    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l" podUID="c119c1b3-bb24-4607-b461-ef4f7f348cbc" containerName="main" containerID="cri-o://07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12" gracePeriod=45
Sep 11 23:27:35.714481 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.714458    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:27:35.765864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.765766    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:35.838080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.838052    2580 generic.go:356] "Generic (PLEG): container finished" podID="c119c1b3-bb24-4607-b461-ef4f7f348cbc" containerID="07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12" exitCode=0
Sep 11 23:27:35.838209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.838087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l" event={"ID":"c119c1b3-bb24-4607-b461-ef4f7f348cbc","Type":"ContainerDied","Data":"07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12"}
Sep 11 23:27:35.838209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.838102    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"
Sep 11 23:27:35.838209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.838113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l" event={"ID":"c119c1b3-bb24-4607-b461-ef4f7f348cbc","Type":"ContainerDied","Data":"cf3847a54fc0e6739630ef9346d12c83fa10165b6f1ea6ea1aebe810221f8423"}
Sep 11 23:27:35.838209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.838129    2580 scope.go:117] "RemoveContainer" containerID="07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12"
Sep 11 23:27:35.846330 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.846313    2580 scope.go:117] "RemoveContainer" containerID="07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12"
Sep 11 23:27:35.846592 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:27:35.846572    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12\": container with ID starting with 07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12 not found: ID does not exist" containerID="07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12"
Sep 11 23:27:35.846645 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.846602    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12"} err="failed to get container status \"07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12\": rpc error: code = NotFound desc = could not find container \"07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12\": container with ID starting with 07df7821024c2dcef1dbf1532d8c297d1ff06e3ebed8d4322910a8ad2a9bdb12 not found: ID does not exist"
Sep 11 23:27:35.877909 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.877887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-dshm\") pod \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") "
Sep 11 23:27:35.878000 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.877937    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tmp-dir\") pod \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") "
Sep 11 23:27:35.878000 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.877983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-model-cache\") pod \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") "
Sep 11 23:27:35.878111 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878019    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-home\") pod \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") "
Sep 11 23:27:35.878111 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878061    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tls-certs\") pod \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") "
Sep 11 23:27:35.878218 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878122    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4wzf\" (UniqueName: \"kubernetes.io/projected/c119c1b3-bb24-4607-b461-ef4f7f348cbc-kube-api-access-t4wzf\") pod \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\" (UID: \"c119c1b3-bb24-4607-b461-ef4f7f348cbc\") "
Sep 11 23:27:35.878272 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878249    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c119c1b3-bb24-4607-b461-ef4f7f348cbc" (UID: "c119c1b3-bb24-4607-b461-ef4f7f348cbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:35.878338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-model-cache" (OuterVolumeSpecName: "model-cache") pod "c119c1b3-bb24-4607-b461-ef4f7f348cbc" (UID: "c119c1b3-bb24-4607-b461-ef4f7f348cbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:35.878338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878274    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-home" (OuterVolumeSpecName: "home") pod "c119c1b3-bb24-4607-b461-ef4f7f348cbc" (UID: "c119c1b3-bb24-4607-b461-ef4f7f348cbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:35.878435 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878379    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:35.878435 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878391    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:35.878435 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.878401    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:35.880047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.880020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-dshm" (OuterVolumeSpecName: "dshm") pod "c119c1b3-bb24-4607-b461-ef4f7f348cbc" (UID: "c119c1b3-bb24-4607-b461-ef4f7f348cbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:27:35.880134 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.880058    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c119c1b3-bb24-4607-b461-ef4f7f348cbc" (UID: "c119c1b3-bb24-4607-b461-ef4f7f348cbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:27:35.880134 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.880075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c119c1b3-bb24-4607-b461-ef4f7f348cbc-kube-api-access-t4wzf" (OuterVolumeSpecName: "kube-api-access-t4wzf") pod "c119c1b3-bb24-4607-b461-ef4f7f348cbc" (UID: "c119c1b3-bb24-4607-b461-ef4f7f348cbc"). InnerVolumeSpecName "kube-api-access-t4wzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:27:35.979271 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.979243    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c119c1b3-bb24-4607-b461-ef4f7f348cbc-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:35.979378 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.979268    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4wzf\" (UniqueName: \"kubernetes.io/projected/c119c1b3-bb24-4607-b461-ef4f7f348cbc-kube-api-access-t4wzf\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:35.979378 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:35.979289    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c119c1b3-bb24-4607-b461-ef4f7f348cbc-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:36.160864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:36.160838    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"]
Sep 11 23:27:36.164909 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:36.164886    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/e2e-conflict-a-8c150bb3-kserve-84994bf775-jt56l"]
Sep 11 23:27:36.510788 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:36.510705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:37.314151 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:37.314116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:37.414374 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:37.414336    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c119c1b3-bb24-4607-b461-ef4f7f348cbc" path="/var/lib/kubelet/pods/c119c1b3-bb24-4607-b461-ef4f7f348cbc/volumes"
Sep 11 23:27:38.164510 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:38.164473    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:39.031027 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:39.030995    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:39.789366 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:39.789333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:40.546442 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:40.546410    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:40.927695 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:40.927662    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:40.928122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:40.927723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:40.940259 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:40.940231    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:41.558076 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:41.558041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:41.880266 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:41.880236    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:27:42.404023 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:42.403987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:51.336802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.336762    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:51.365838 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.365805    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"]
Sep 11 23:27:51.366108 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.366084    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2" podUID="3345f82a-acf2-4595-95af-a7da4f591c99" containerName="payload-processing" containerID="cri-o://e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb" gracePeriod=30
Sep 11 23:27:51.435227 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.435196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:51.534535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.534497    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:51.619871 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.619845    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:27:51.634579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.634540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:51.711187 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.711157    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3345f82a-acf2-4595-95af-a7da4f591c99-plugins-config-volume\") pod \"3345f82a-acf2-4595-95af-a7da4f591c99\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") "
Sep 11 23:27:51.711531 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.711508    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3345f82a-acf2-4595-95af-a7da4f591c99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3345f82a-acf2-4595-95af-a7da4f591c99" (UID: "3345f82a-acf2-4595-95af-a7da4f591c99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:27:51.811682 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.811650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nqcz\" (UniqueName: \"kubernetes.io/projected/3345f82a-acf2-4595-95af-a7da4f591c99-kube-api-access-4nqcz\") pod \"3345f82a-acf2-4595-95af-a7da4f591c99\" (UID: \"3345f82a-acf2-4595-95af-a7da4f591c99\") "
Sep 11 23:27:51.811875 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.811863    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3345f82a-acf2-4595-95af-a7da4f591c99-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:51.813666 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.813631    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3345f82a-acf2-4595-95af-a7da4f591c99-kube-api-access-4nqcz" (OuterVolumeSpecName: "kube-api-access-4nqcz") pod "3345f82a-acf2-4595-95af-a7da4f591c99" (UID: "3345f82a-acf2-4595-95af-a7da4f591c99"). InnerVolumeSpecName "kube-api-access-4nqcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:27:51.912717 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.912647    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nqcz\" (UniqueName: \"kubernetes.io/projected/3345f82a-acf2-4595-95af-a7da4f591c99-kube-api-access-4nqcz\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:27:51.928063 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.928027    2580 generic.go:356] "Generic (PLEG): container finished" podID="3345f82a-acf2-4595-95af-a7da4f591c99" containerID="e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb" exitCode=0
Sep 11 23:27:51.928207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.928073    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"
Sep 11 23:27:51.928207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.928065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2" event={"ID":"3345f82a-acf2-4595-95af-a7da4f591c99","Type":"ContainerDied","Data":"e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb"}
Sep 11 23:27:51.928207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.928198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2" event={"ID":"3345f82a-acf2-4595-95af-a7da4f591c99","Type":"ContainerDied","Data":"2733db3bbc733427519f03cb454d7c0f1d9e8f7e445b53ee9a6d6e181231b24a"}
Sep 11 23:27:51.928377 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.928230    2580 scope.go:117] "RemoveContainer" containerID="e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb"
Sep 11 23:27:51.937126 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.937111    2580 scope.go:117] "RemoveContainer" containerID="e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb"
Sep 11 23:27:51.937398 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:27:51.937381    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb\": container with ID starting with e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb not found: ID does not exist" containerID="e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb"
Sep 11 23:27:51.937459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.937409    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb"} err="failed to get container status \"e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb\": rpc error: code = NotFound desc = could not find container \"e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb\": container with ID starting with e2e04d4ffefe9f7a4a5287f3c8dad0e29ccf2658fc37753a04418a6b08df00cb not found: ID does not exist"
Sep 11 23:27:51.953669 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.953639    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"]
Sep 11 23:27:51.960191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:51.960171    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b96fb0e1-5ccd848db5-pj5s2"]
Sep 11 23:27:52.071205 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:52.071170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:52.976749 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:52.976709    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:53.414398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:53.414365    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3345f82a-acf2-4595-95af-a7da4f591c99" path="/var/lib/kubelet/pods/3345f82a-acf2-4595-95af-a7da4f591c99/volumes"
Sep 11 23:27:53.853614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:53.853579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:54.565052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:54.565020    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:54.665650 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:54.665614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:55.409033 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:55.408988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:56.258214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:56.258177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:56.976942 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:56.976897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:27:59.640231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:27:59.640192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:28:06.905991 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.905956    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"]
Sep 11 23:28:06.906447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.906344    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3345f82a-acf2-4595-95af-a7da4f591c99" containerName="payload-processing"
Sep 11 23:28:06.906447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.906355    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3345f82a-acf2-4595-95af-a7da4f591c99" containerName="payload-processing"
Sep 11 23:28:06.906447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.906374    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c119c1b3-bb24-4607-b461-ef4f7f348cbc" containerName="main"
Sep 11 23:28:06.906447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.906380    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c119c1b3-bb24-4607-b461-ef4f7f348cbc" containerName="main"
Sep 11 23:28:06.906578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.906456    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c119c1b3-bb24-4607-b461-ef4f7f348cbc" containerName="main"
Sep 11 23:28:06.906578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.906469    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3345f82a-acf2-4595-95af-a7da4f591c99" containerName="payload-processing"
Sep 11 23:28:06.909528 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.909511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:06.913993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.913964    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-da9451\"/\"openshift-service-ca.crt\""
Sep 11 23:28:06.913993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.913986    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-da9451\"/\"auth-test-model-da9451-kserve-self-signed-certs\""
Sep 11 23:28:06.913993 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.913969    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-da9451\"/\"kube-root-ca.crt\""
Sep 11 23:28:06.914262 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.913991    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-da9451\"/\"default-dockercfg-8dm8j\""
Sep 11 23:28:06.920292 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.920271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"]
Sep 11 23:28:06.937059 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.937033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-tmp-dir\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:06.937172 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.937079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-model-cache\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:06.937172 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.937102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2be8496-11c2-4253-9f68-5f05c36211fd-tls-certs\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:06.937252 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.937177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-home\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:06.937252 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.937242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-dshm\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:06.937324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:06.937262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz7tr\" (UniqueName: \"kubernetes.io/projected/a2be8496-11c2-4253-9f68-5f05c36211fd-kube-api-access-vz7tr\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.038656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.038628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-tmp-dir\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.038802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.038672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-model-cache\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.038802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.038696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2be8496-11c2-4253-9f68-5f05c36211fd-tls-certs\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.038802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.038720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-home\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.038802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.038763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-dshm\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.038802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.038789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz7tr\" (UniqueName: \"kubernetes.io/projected/a2be8496-11c2-4253-9f68-5f05c36211fd-kube-api-access-vz7tr\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.039138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.039105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-tmp-dir\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.039138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.039119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-model-cache\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.039238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.039170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-home\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.041089 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.041064    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-dshm\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.041259 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.041244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2be8496-11c2-4253-9f68-5f05c36211fd-tls-certs\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.048784 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.048751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz7tr\" (UniqueName: \"kubernetes.io/projected/a2be8496-11c2-4253-9f68-5f05c36211fd-kube-api-access-vz7tr\") pod \"auth-test-model-da9451-kserve-df85cd7f5-695nd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") " pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.114322 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.114292    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:28:07.219273 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.219199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:07.490451 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:07.490421    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"]
Sep 11 23:28:07.492373 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:28:07.492345    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2be8496_11c2_4253_9f68_5f05c36211fd.slice/crio-405d922be09bb1999d2059e5ad74567885639a52a372e219a20177296b7bccaa WatchSource:0}: Error finding container 405d922be09bb1999d2059e5ad74567885639a52a372e219a20177296b7bccaa: Status 404 returned error can't find the container with id 405d922be09bb1999d2059e5ad74567885639a52a372e219a20177296b7bccaa
Sep 11 23:28:08.008323 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:08.008282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd" event={"ID":"a2be8496-11c2-4253-9f68-5f05c36211fd","Type":"ContainerStarted","Data":"ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c"}
Sep 11 23:28:08.008724 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:08.008331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd" event={"ID":"a2be8496-11c2-4253-9f68-5f05c36211fd","Type":"ContainerStarted","Data":"405d922be09bb1999d2059e5ad74567885639a52a372e219a20177296b7bccaa"}
Sep 11 23:28:08.025608 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:08.025560    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd" podStartSLOduration=2.025546343 podStartE2EDuration="2.025546343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:28:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:28:08.024199861 +0000 UTC m=+1603.236171036" watchObservedRunningTime="2026-09-11 23:28:08.025546343 +0000 UTC m=+1603.237517516"
Sep 11 23:28:17.219731 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:17.219693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:17.220153 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:17.219748    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:17.232365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:17.232340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:18.070352 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:18.070325    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:28:21.015096 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:21.015053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:28:35.682832 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:35.682795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:28:36.540170 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:36.540124    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:28:42.142176 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:42.142139    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:28:50.342634 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:28:50.342589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:04.541809 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:04.541769    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:06.218322 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:06.218275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:06.364695 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:06.364665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:06.502607 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:06.502515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:07.725378 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:07.725342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:07.852401 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:07.852355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:08.646100 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:08.646068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:08.827732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:08.827694    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:10.084617 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:10.084577    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:12.558545 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:12.558511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:16.169165 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:16.169127    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:16.254676 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:16.254637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:16.373252 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:16.373215    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:16.932087 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:16.932052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:24.092859 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:24.092826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:24.203096 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:24.203062    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:24.291936 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:24.291880    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:25.643219 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:25.643185    2580 scope.go:117] "RemoveContainer" containerID="745d75f3a80d7b7a00f0ac7285959aade2f3f65b99a31bfa37e5162c23d1626d"
Sep 11 23:29:28.152932 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:28.152883    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:34.778100 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:34.778060    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:34.996571 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:34.996536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:40.679130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:40.679097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:40.779816 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:40.779775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:46.935358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:46.935318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:52.914268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:52.914182    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:29:55.887158 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:29:55.887123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:00.148653 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.148605    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"]
Sep 11 23:30:00.151816 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.151797    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"]
Sep 11 23:30:00.151962 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.151942    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"
Sep 11 23:30:00.155127 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.155105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"
Sep 11 23:30:00.155487 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.155472    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vrx7m\""
Sep 11 23:30:00.163809 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.163786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"
Sep 11 23:30:00.169935 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.169896    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"]
Sep 11 23:30:00.173085 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.173058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"
Sep 11 23:30:00.175792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.175757    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"]
Sep 11 23:30:00.179805 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.179782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"
Sep 11 23:30:00.192473 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.191497    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"]
Sep 11 23:30:00.193455 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.193278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"]
Sep 11 23:30:00.224983 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.224961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"
Sep 11 23:30:00.453520 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.453493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"]
Sep 11 23:30:00.455478 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:30:00.455453    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a7aacc_5ed4_4e5d_aed0_7dd3b0aabb37.slice/crio-5a9a0bed3d3579e7bbd009d192db9eeb8b5a764a046feb1e2302904ce1a6a936 WatchSource:0}: Error finding container 5a9a0bed3d3579e7bbd009d192db9eeb8b5a764a046feb1e2302904ce1a6a936: Status 404 returned error can't find the container with id 5a9a0bed3d3579e7bbd009d192db9eeb8b5a764a046feb1e2302904ce1a6a936
Sep 11 23:30:00.474882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.474803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"]
Sep 11 23:30:00.476316 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:30:00.476289    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3efb1017_78f3_4e20_9a97_9a1ca2680f2a.slice/crio-738105d426cab7d1788692f6416b0c77b71442ef4b013397818b774c70a0adce WatchSource:0}: Error finding container 738105d426cab7d1788692f6416b0c77b71442ef4b013397818b774c70a0adce: Status 404 returned error can't find the container with id 738105d426cab7d1788692f6416b0c77b71442ef4b013397818b774c70a0adce
Sep 11 23:30:00.525437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.525413    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"]
Sep 11 23:30:00.527212 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:30:00.527180    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807f6a14_6862_42a9_a0b0_b2b1b8f20643.slice/crio-59e19225b8f352581a74bcabd6560633349e03fcad2276008e807895202311f3 WatchSource:0}: Error finding container 59e19225b8f352581a74bcabd6560633349e03fcad2276008e807895202311f3: Status 404 returned error can't find the container with id 59e19225b8f352581a74bcabd6560633349e03fcad2276008e807895202311f3
Sep 11 23:30:00.563622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.563584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8" event={"ID":"57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37","Type":"ContainerStarted","Data":"5a9a0bed3d3579e7bbd009d192db9eeb8b5a764a046feb1e2302904ce1a6a936"}
Sep 11 23:30:00.565229 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.565202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh" event={"ID":"3efb1017-78f3-4e20-9a97-9a1ca2680f2a","Type":"ContainerStarted","Data":"738105d426cab7d1788692f6416b0c77b71442ef4b013397818b774c70a0adce"}
Sep 11 23:30:00.566521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:00.566497    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln" event={"ID":"807f6a14-6862-42a9-a0b0-b2b1b8f20643","Type":"ContainerStarted","Data":"59e19225b8f352581a74bcabd6560633349e03fcad2276008e807895202311f3"}
Sep 11 23:30:01.574214 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:01.574170    2580 generic.go:356] "Generic (PLEG): container finished" podID="3efb1017-78f3-4e20-9a97-9a1ca2680f2a" containerID="321d4421f36586300a914e435285ef70ba8e9ca786ec5992142d9ac82f603fa1" exitCode=0
Sep 11 23:30:01.574645 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:01.574250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh" event={"ID":"3efb1017-78f3-4e20-9a97-9a1ca2680f2a","Type":"ContainerDied","Data":"321d4421f36586300a914e435285ef70ba8e9ca786ec5992142d9ac82f603fa1"}
Sep 11 23:30:01.575824 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:01.575801    2580 generic.go:356] "Generic (PLEG): container finished" podID="807f6a14-6862-42a9-a0b0-b2b1b8f20643" containerID="7f76f67d6ed0593a414bdfdcbc37ace0f9593204374374e34624bd114f11cc69" exitCode=0
Sep 11 23:30:01.575928 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:01.575869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln" event={"ID":"807f6a14-6862-42a9-a0b0-b2b1b8f20643","Type":"ContainerDied","Data":"7f76f67d6ed0593a414bdfdcbc37ace0f9593204374374e34624bd114f11cc69"}
Sep 11 23:30:01.577507 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:01.577482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8" event={"ID":"57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37","Type":"ContainerStarted","Data":"ac19708a7873449a9cf2e7a7fcdb0c8b2b3dd5a5df758c90a2b2faebec5a0934"}
Sep 11 23:30:01.608768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:01.608725    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8" podStartSLOduration=1.608708317 podStartE2EDuration="1.608708317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:30:01.608301256 +0000 UTC m=+1716.820272432" watchObservedRunningTime="2026-09-11 23:30:01.608708317 +0000 UTC m=+1716.820679533"
Sep 11 23:30:02.209429 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:02.209396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:02.484205 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:02.484125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:02.584703 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:02.584668    2580 generic.go:356] "Generic (PLEG): container finished" podID="57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37" containerID="ac19708a7873449a9cf2e7a7fcdb0c8b2b3dd5a5df758c90a2b2faebec5a0934" exitCode=0
Sep 11 23:30:02.585166 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:02.584801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8" event={"ID":"57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37","Type":"ContainerDied","Data":"ac19708a7873449a9cf2e7a7fcdb0c8b2b3dd5a5df758c90a2b2faebec5a0934"}
Sep 11 23:30:02.597644 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:02.597593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:02.717675 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:02.717648    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"
Sep 11 23:30:02.752646 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:02.752620    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"
Sep 11 23:30:03.069568 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.069536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:03.431516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.431485    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:03.591611 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.591581    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"
Sep 11 23:30:03.592051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.591636    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln" event={"ID":"807f6a14-6862-42a9-a0b0-b2b1b8f20643","Type":"ContainerDied","Data":"59e19225b8f352581a74bcabd6560633349e03fcad2276008e807895202311f3"}
Sep 11 23:30:03.592051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.591674    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59e19225b8f352581a74bcabd6560633349e03fcad2276008e807895202311f3"
Sep 11 23:30:03.593000 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.592984    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"
Sep 11 23:30:03.593103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.593021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh" event={"ID":"3efb1017-78f3-4e20-9a97-9a1ca2680f2a","Type":"ContainerDied","Data":"738105d426cab7d1788692f6416b0c77b71442ef4b013397818b774c70a0adce"}
Sep 11 23:30:03.593103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.593055    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="738105d426cab7d1788692f6416b0c77b71442ef4b013397818b774c70a0adce"
Sep 11 23:30:03.731516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:03.731492    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"
Sep 11 23:30:04.600113 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:04.600076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8" event={"ID":"57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37","Type":"ContainerDied","Data":"5a9a0bed3d3579e7bbd009d192db9eeb8b5a764a046feb1e2302904ce1a6a936"}
Sep 11 23:30:04.600113 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:04.600096    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"
Sep 11 23:30:04.600564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:04.600132    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a9a0bed3d3579e7bbd009d192db9eeb8b5a764a046feb1e2302904ce1a6a936"
Sep 11 23:30:05.701140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:05.701100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:05.817657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:05.817618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:05.980540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:05.980453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:07.186353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:07.186321    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:08.184206 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:08.184178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:09.069532 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:09.069496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:09.908728 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:09.908697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:10.732234 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:10.732202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:11.694331 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:11.694290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:11.820490 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:11.820456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:12.211788 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.211758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:12.310391 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.310345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:12.414541 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.414495    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"]
Sep 11 23:30:12.415013 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.414995    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3efb1017-78f3-4e20-9a97-9a1ca2680f2a" containerName="cleanup"
Sep 11 23:30:12.415110 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415014    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3efb1017-78f3-4e20-9a97-9a1ca2680f2a" containerName="cleanup"
Sep 11 23:30:12.415110 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415030    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37" containerName="cleanup"
Sep 11 23:30:12.415110 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415038    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37" containerName="cleanup"
Sep 11 23:30:12.415110 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415066    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807f6a14-6862-42a9-a0b0-b2b1b8f20643" containerName="cleanup"
Sep 11 23:30:12.415110 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415075    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="807f6a14-6862-42a9-a0b0-b2b1b8f20643" containerName="cleanup"
Sep 11 23:30:12.415358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415169    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37" containerName="cleanup"
Sep 11 23:30:12.415358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415184    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="807f6a14-6862-42a9-a0b0-b2b1b8f20643" containerName="cleanup"
Sep 11 23:30:12.415358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.415194    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3efb1017-78f3-4e20-9a97-9a1ca2680f2a" containerName="cleanup"
Sep 11 23:30:12.418044 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.418015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:12.418455 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.418437    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:12.424747 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.424726    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-451902-dockercfg-9rg9d\""
Sep 11 23:30:12.425270 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.425021    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-451902\""
Sep 11 23:30:12.447760 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.447727    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"]
Sep 11 23:30:12.567881 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.567847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55bg6\" (UniqueName: \"kubernetes.io/projected/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-kube-api-access-55bg6\") pod \"payload-processing-e2e-api-a-451902-7fc87746df-cvv8t\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:12.568072 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.567939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-451902-7fc87746df-cvv8t\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:12.669301 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.669265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-451902-7fc87746df-cvv8t\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:12.669500 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.669346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55bg6\" (UniqueName: \"kubernetes.io/projected/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-kube-api-access-55bg6\") pod \"payload-processing-e2e-api-a-451902-7fc87746df-cvv8t\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:12.670018 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.669999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-451902-7fc87746df-cvv8t\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:12.680887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.680860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55bg6\" (UniqueName: \"kubernetes.io/projected/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-kube-api-access-55bg6\") pod \"payload-processing-e2e-api-a-451902-7fc87746df-cvv8t\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:12.728968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:12.728896    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:13.004356 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:13.004323    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"]
Sep 11 23:30:13.006052 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:30:13.006026    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39975ea3_fa00_4ae0_adf0_b15e9821ccd4.slice/crio-feba72532ed600e31c3e0b10789cff17cca6ba397a6d01e19effc17de65e25ee WatchSource:0}: Error finding container feba72532ed600e31c3e0b10789cff17cca6ba397a6d01e19effc17de65e25ee: Status 404 returned error can't find the container with id feba72532ed600e31c3e0b10789cff17cca6ba397a6d01e19effc17de65e25ee
Sep 11 23:30:13.647400 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:13.647362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t" event={"ID":"39975ea3-fa00-4ae0-adf0-b15e9821ccd4","Type":"ContainerStarted","Data":"5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e"}
Sep 11 23:30:13.647569 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:13.647412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t" event={"ID":"39975ea3-fa00-4ae0-adf0-b15e9821ccd4","Type":"ContainerStarted","Data":"feba72532ed600e31c3e0b10789cff17cca6ba397a6d01e19effc17de65e25ee"}
Sep 11 23:30:13.647569 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:13.647490    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:13.669933 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:13.669849    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t" podStartSLOduration=1.669829298 podStartE2EDuration="1.669829298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:30:13.66713755 +0000 UTC m=+1728.879108724" watchObservedRunningTime="2026-09-11 23:30:13.669829298 +0000 UTC m=+1728.881800493"
Sep 11 23:30:24.654992 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:24.654953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:30:31.264579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:31.264527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:31.589840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:31.589803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:32.573521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:32.573486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:33.675540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:33.675504    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:34.624326 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:34.624290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:35.384826 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:35.384791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:35.697379 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:35.697296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:36.131699 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:36.131656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:36.346850 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:36.346814    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:36.451211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:36.451121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:36.558877 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:36.558837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:36.758316 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:36.758187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:37.421656 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:37.421626    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:38.479774 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:38.479736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:39.487684 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:39.487643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:39.865450 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:39.865416    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:39.966900 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:39.966869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:40.424879 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:40.424846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:41.420892 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:41.420861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:42.352601 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:42.352568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:43.194769 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.194735    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:43.294519 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.294478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:43.718506 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.718474    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:43.820331 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.820298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:43.845387 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.845356    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"]
Sep 11 23:30:43.849211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.849194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:43.852037 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.852013    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d1b012-dockercfg-9hzm8\""
Sep 11 23:30:43.852358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.852345    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d1b012\""
Sep 11 23:30:43.863524 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.863500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"]
Sep 11 23:30:43.934704 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.934667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:43.949057 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.949022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") " pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:43.949225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:43.949152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58j8z\" (UniqueName: \"kubernetes.io/projected/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-kube-api-access-58j8z\") pod \"payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") " pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:44.050753 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.050655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") " pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:44.050907 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.050757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58j8z\" (UniqueName: \"kubernetes.io/projected/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-kube-api-access-58j8z\") pod \"payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") " pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:44.051425 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.051406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") " pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:44.059507 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.059478    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58j8z\" (UniqueName: \"kubernetes.io/projected/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-kube-api-access-58j8z\") pod \"payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") " pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:44.159061 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.159027    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:44.163604 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.163576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:44.433763 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.433734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"]
Sep 11 23:30:44.436351 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:30:44.436328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5b0ddc_f4fc_4f4c_81f1_ce3bf6b453a0.slice/crio-4d0fa19b2930b0a427bca325745715b1a00d77650a32e6748a1160f6dc1c8fa3 WatchSource:0}: Error finding container 4d0fa19b2930b0a427bca325745715b1a00d77650a32e6748a1160f6dc1c8fa3: Status 404 returned error can't find the container with id 4d0fa19b2930b0a427bca325745715b1a00d77650a32e6748a1160f6dc1c8fa3
Sep 11 23:30:44.801562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.801465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2" event={"ID":"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0","Type":"ContainerStarted","Data":"d7e5f8ef0969f01c6c3e7ddf40a72b0dc0fd835f282046f5a794464bbfce13c1"}
Sep 11 23:30:44.801562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.801522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2" event={"ID":"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0","Type":"ContainerStarted","Data":"4d0fa19b2930b0a427bca325745715b1a00d77650a32e6748a1160f6dc1c8fa3"}
Sep 11 23:30:44.801562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.801553    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:44.822815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:44.822761    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2" podStartSLOduration=1.822745295 podStartE2EDuration="1.822745295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:30:44.819816316 +0000 UTC m=+1760.031787503" watchObservedRunningTime="2026-09-11 23:30:44.822745295 +0000 UTC m=+1760.034716521"
Sep 11 23:30:45.122603 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:45.122570    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:46.063826 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:46.063791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:46.821244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:46.821212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:47.672094 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:47.672054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:52.575457 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.575428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:52.602240 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.602203    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"]
Sep 11 23:30:52.602499 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.602472    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8" podUID="105e5c69-f920-4ccb-9659-140d0bf31b72" containerName="payload-processing" containerID="cri-o://ba4efe22f10fe4a744bb2e6c3f54b706cfa6c26d08a17ed7bdf8837cffe8ca69" gracePeriod=30
Sep 11 23:30:52.650141 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.650106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:52.750876 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.750846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:52.830618 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.830533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:52.846407 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.846376    2580 generic.go:356] "Generic (PLEG): container finished" podID="105e5c69-f920-4ccb-9659-140d0bf31b72" containerID="ba4efe22f10fe4a744bb2e6c3f54b706cfa6c26d08a17ed7bdf8837cffe8ca69" exitCode=0
Sep 11 23:30:52.846544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.846454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8" event={"ID":"105e5c69-f920-4ccb-9659-140d0bf31b72","Type":"ContainerDied","Data":"ba4efe22f10fe4a744bb2e6c3f54b706cfa6c26d08a17ed7bdf8837cffe8ca69"}
Sep 11 23:30:52.862826 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.862800    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:30:52.933284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.933240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzmbq\" (UniqueName: \"kubernetes.io/projected/105e5c69-f920-4ccb-9659-140d0bf31b72-kube-api-access-pzmbq\") pod \"105e5c69-f920-4ccb-9659-140d0bf31b72\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") "
Sep 11 23:30:52.933479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.933307    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/105e5c69-f920-4ccb-9659-140d0bf31b72-plugins-config-volume\") pod \"105e5c69-f920-4ccb-9659-140d0bf31b72\" (UID: \"105e5c69-f920-4ccb-9659-140d0bf31b72\") "
Sep 11 23:30:52.933649 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.933625    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105e5c69-f920-4ccb-9659-140d0bf31b72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "105e5c69-f920-4ccb-9659-140d0bf31b72" (UID: "105e5c69-f920-4ccb-9659-140d0bf31b72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:30:52.935366 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:52.935341    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/105e5c69-f920-4ccb-9659-140d0bf31b72-kube-api-access-pzmbq" (OuterVolumeSpecName: "kube-api-access-pzmbq") pod "105e5c69-f920-4ccb-9659-140d0bf31b72" (UID: "105e5c69-f920-4ccb-9659-140d0bf31b72"). InnerVolumeSpecName "kube-api-access-pzmbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:30:53.034787 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.034743    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/105e5c69-f920-4ccb-9659-140d0bf31b72-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:30:53.034787 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.034779    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzmbq\" (UniqueName: \"kubernetes.io/projected/105e5c69-f920-4ccb-9659-140d0bf31b72-kube-api-access-pzmbq\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:30:53.141543 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.141453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:53.365458 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.365426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:53.853099 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.853062    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"
Sep 11 23:30:53.853566 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.853070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8" event={"ID":"105e5c69-f920-4ccb-9659-140d0bf31b72","Type":"ContainerDied","Data":"d33480463720e95dd626fe51dde36911746a84b4994f494741f077b6fe3390b3"}
Sep 11 23:30:53.853566 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.853212    2580 scope.go:117] "RemoveContainer" containerID="ba4efe22f10fe4a744bb2e6c3f54b706cfa6c26d08a17ed7bdf8837cffe8ca69"
Sep 11 23:30:53.871826 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.871796    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"]
Sep 11 23:30:53.879472 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:53.879445    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-12fb6b-5ff4c775b5-kbtz8"]
Sep 11 23:30:54.242619 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:54.242523    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:55.035307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:55.035272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:55.414592 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:55.414558    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="105e5c69-f920-4ccb-9659-140d0bf31b72" path="/var/lib/kubelet/pods/105e5c69-f920-4ccb-9659-140d0bf31b72/volumes"
Sep 11 23:30:55.810979 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:55.810861    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:30:56.066521 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:56.066488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:57.027221 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:57.027184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:57.959179 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:57.959147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:58.771262 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:58.771228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:59.558644 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:59.558606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:59.659291 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:59.659256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:59.816449 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:59.816416    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:30:59.917899 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:30:59.917867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:00.096191 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:00.096098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:00.383511 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:00.383432    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:01.488312 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:01.488277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:02.803940 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:02.803877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:03.871053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:03.871022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:04.873528 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:04.873492    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:05.835622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:05.835587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:06.997123 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:06.997083    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:07.766702 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:07.766664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:08.626393 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:08.626358    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:09.494513 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:09.494475    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:10.385936 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:10.385874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:12.752727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:12.752696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:15.327068 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:15.327023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:15.426864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:15.426830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:15.909393 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:15.909357    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:18.465540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.465462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:18.496018 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.495981    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"]
Sep 11 23:31:18.496284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.496254    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t" podUID="39975ea3-fa00-4ae0-adf0-b15e9821ccd4" containerName="payload-processing" containerID="cri-o://5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e" gracePeriod=30
Sep 11 23:31:18.567840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.567790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:18.670212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.670179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:18.761219 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.761187    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:31:18.769723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.769699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:18.867200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.867163    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55bg6\" (UniqueName: \"kubernetes.io/projected/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-kube-api-access-55bg6\") pod \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") "
Sep 11 23:31:18.867401 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.867265    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-plugins-config-volume\") pod \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\" (UID: \"39975ea3-fa00-4ae0-adf0-b15e9821ccd4\") "
Sep 11 23:31:18.867642 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.867616    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39975ea3-fa00-4ae0-adf0-b15e9821ccd4" (UID: "39975ea3-fa00-4ae0-adf0-b15e9821ccd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:31:18.869270 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.869241    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-kube-api-access-55bg6" (OuterVolumeSpecName: "kube-api-access-55bg6") pod "39975ea3-fa00-4ae0-adf0-b15e9821ccd4" (UID: "39975ea3-fa00-4ae0-adf0-b15e9821ccd4"). InnerVolumeSpecName "kube-api-access-55bg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:31:18.968229 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.968170    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:31:18.968229 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.968219    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55bg6\" (UniqueName: \"kubernetes.io/projected/39975ea3-fa00-4ae0-adf0-b15e9821ccd4-kube-api-access-55bg6\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:31:18.983483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.983402    2580 generic.go:356] "Generic (PLEG): container finished" podID="39975ea3-fa00-4ae0-adf0-b15e9821ccd4" containerID="5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e" exitCode=0
Sep 11 23:31:18.983483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.983447    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"
Sep 11 23:31:18.983483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.983468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t" event={"ID":"39975ea3-fa00-4ae0-adf0-b15e9821ccd4","Type":"ContainerDied","Data":"5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e"}
Sep 11 23:31:18.983727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.983519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t" event={"ID":"39975ea3-fa00-4ae0-adf0-b15e9821ccd4","Type":"ContainerDied","Data":"feba72532ed600e31c3e0b10789cff17cca6ba397a6d01e19effc17de65e25ee"}
Sep 11 23:31:18.983727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.983548    2580 scope.go:117] "RemoveContainer" containerID="5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e"
Sep 11 23:31:18.992501 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.992479    2580 scope.go:117] "RemoveContainer" containerID="5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e"
Sep 11 23:31:18.992795 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:31:18.992773    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e\": container with ID starting with 5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e not found: ID does not exist" containerID="5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e"
Sep 11 23:31:18.992842 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:18.992806    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e"} err="failed to get container status \"5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e\": rpc error: code = NotFound desc = could not find container \"5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e\": container with ID starting with 5984e8bdc4c5191ab60cff441b44ac7adf74fc9005f6e7b923cb88a21e713c1e not found: ID does not exist"
Sep 11 23:31:19.010432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:19.010392    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"]
Sep 11 23:31:19.017083 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:19.017056    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-451902-7fc87746df-cvv8t"]
Sep 11 23:31:19.097772 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:19.097739    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:19.242974 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:19.242872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:19.413743 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:19.413710    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39975ea3-fa00-4ae0-adf0-b15e9821ccd4" path="/var/lib/kubelet/pods/39975ea3-fa00-4ae0-adf0-b15e9821ccd4/volumes"
Sep 11 23:31:20.330435 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:20.330399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:21.090507 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:21.090469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:22.182160 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:22.182114    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:22.891663 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:22.891620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:23.755093 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:23.755001    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:24.675712 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:24.675678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:25.556747 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:25.556710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:25.657551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:25.657516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:25.819154 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:25.819121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:26.318324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:26.318293    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:27.195411 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:27.195364    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:28.048653 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:28.048611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:29.083074 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:29.083037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:29.865052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:29.865017    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:30.846479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:30.846439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:31.741238 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:31.741190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:32.775201 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:32.775171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:38.652451 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:38.652407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:41.743720 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:41.743672    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:41.905381 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:41.905327    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"]
Sep 11 23:31:41.905617 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:41.905587    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2" podUID="2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" containerName="payload-processing" containerID="cri-o://d7e5f8ef0969f01c6c3e7ddf40a72b0dc0fd835f282046f5a794464bbfce13c1" gracePeriod=30
Sep 11 23:31:42.107149 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.107112    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" containerID="d7e5f8ef0969f01c6c3e7ddf40a72b0dc0fd835f282046f5a794464bbfce13c1" exitCode=0
Sep 11 23:31:42.107331 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.107180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2" event={"ID":"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0","Type":"ContainerDied","Data":"d7e5f8ef0969f01c6c3e7ddf40a72b0dc0fd835f282046f5a794464bbfce13c1"}
Sep 11 23:31:42.167341 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.167314    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:31:42.289727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.289624    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-plugins-config-volume\") pod \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") "
Sep 11 23:31:42.289727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.289681    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58j8z\" (UniqueName: \"kubernetes.io/projected/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-kube-api-access-58j8z\") pod \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\" (UID: \"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0\") "
Sep 11 23:31:42.290081 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.290057    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" (UID: "2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:31:42.291852 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.291826    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-kube-api-access-58j8z" (OuterVolumeSpecName: "kube-api-access-58j8z") pod "2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" (UID: "2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0"). InnerVolumeSpecName "kube-api-access-58j8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:31:42.390636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.390588    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:31:42.390636 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:42.390625    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58j8z\" (UniqueName: \"kubernetes.io/projected/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0-kube-api-access-58j8z\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:31:43.113551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.113517    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"
Sep 11 23:31:43.113551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.113537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2" event={"ID":"2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0","Type":"ContainerDied","Data":"4d0fa19b2930b0a427bca325745715b1a00d77650a32e6748a1160f6dc1c8fa3"}
Sep 11 23:31:43.114076 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.113632    2580 scope.go:117] "RemoveContainer" containerID="d7e5f8ef0969f01c6c3e7ddf40a72b0dc0fd835f282046f5a794464bbfce13c1"
Sep 11 23:31:43.140804 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.140771    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"]
Sep 11 23:31:43.145727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.145697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:43.146649 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.146631    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d1b012-795cfd6c6f-q58q2"]
Sep 11 23:31:43.414583 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.414495    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" path="/var/lib/kubelet/pods/2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0/volumes"
Sep 11 23:31:43.496533 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:43.496500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:44.456055 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:44.456026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:45.367360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:45.367329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:46.283707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:46.283673    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:47.068532 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:47.068503    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:48.748925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:48.748884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:48.848820 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:48.848781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:49.028516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:49.028426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:49.486598 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:49.486564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:50.486038 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:50.486007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:51.406187 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:51.406149    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:52.278375 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:52.278346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:53.271660 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:53.271627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:53.899825 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:53.899789    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:54.001052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:54.001020    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:54.100135 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:54.100096    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:31:54.930978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:31:54.930944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:00.086658 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.086617    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:00.127988 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.127945    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"]
Sep 11 23:32:00.128232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.128201    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4" podUID="890bb418-26ac-4a5a-9abe-99de37cbccf3" containerName="payload-processing" containerID="cri-o://1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b" gracePeriod=30
Sep 11 23:32:00.136268 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.136012    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"]
Sep 11 23:32:00.136357 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.136242    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx" podUID="de2d5069-c1f7-4362-a802-ece4efcfbf3b" containerName="payload-pre-processing" containerID="cri-o://153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae" gracePeriod=30
Sep 11 23:32:00.174237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.174200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:00.240778 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.240743    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:00.336197 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.336166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:00.408840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.408807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:00.412853 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.412831    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:32:00.415845 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.415814    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:32:00.451707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.451670    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjkvw\" (UniqueName: \"kubernetes.io/projected/890bb418-26ac-4a5a-9abe-99de37cbccf3-kube-api-access-sjkvw\") pod \"890bb418-26ac-4a5a-9abe-99de37cbccf3\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") "
Sep 11 23:32:00.451905 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.451721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/890bb418-26ac-4a5a-9abe-99de37cbccf3-plugins-config-volume\") pod \"890bb418-26ac-4a5a-9abe-99de37cbccf3\" (UID: \"890bb418-26ac-4a5a-9abe-99de37cbccf3\") "
Sep 11 23:32:00.451905 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.451770    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de2d5069-c1f7-4362-a802-ece4efcfbf3b-plugins-config-volume\") pod \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") "
Sep 11 23:32:00.452316 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.452273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/890bb418-26ac-4a5a-9abe-99de37cbccf3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "890bb418-26ac-4a5a-9abe-99de37cbccf3" (UID: "890bb418-26ac-4a5a-9abe-99de37cbccf3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:00.452426 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.452320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2d5069-c1f7-4362-a802-ece4efcfbf3b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de2d5069-c1f7-4362-a802-ece4efcfbf3b" (UID: "de2d5069-c1f7-4362-a802-ece4efcfbf3b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:00.452653 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.452629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm7dh\" (UniqueName: \"kubernetes.io/projected/de2d5069-c1f7-4362-a802-ece4efcfbf3b-kube-api-access-nm7dh\") pod \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\" (UID: \"de2d5069-c1f7-4362-a802-ece4efcfbf3b\") "
Sep 11 23:32:00.453048 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.453025    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/890bb418-26ac-4a5a-9abe-99de37cbccf3-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.453129 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.453051    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de2d5069-c1f7-4362-a802-ece4efcfbf3b-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.455098 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.455054    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/890bb418-26ac-4a5a-9abe-99de37cbccf3-kube-api-access-sjkvw" (OuterVolumeSpecName: "kube-api-access-sjkvw") pod "890bb418-26ac-4a5a-9abe-99de37cbccf3" (UID: "890bb418-26ac-4a5a-9abe-99de37cbccf3"). InnerVolumeSpecName "kube-api-access-sjkvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:00.455307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.455279    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2d5069-c1f7-4362-a802-ece4efcfbf3b-kube-api-access-nm7dh" (OuterVolumeSpecName: "kube-api-access-nm7dh") pod "de2d5069-c1f7-4362-a802-ece4efcfbf3b" (UID: "de2d5069-c1f7-4362-a802-ece4efcfbf3b"). InnerVolumeSpecName "kube-api-access-nm7dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:00.554232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.554193    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm7dh\" (UniqueName: \"kubernetes.io/projected/de2d5069-c1f7-4362-a802-ece4efcfbf3b-kube-api-access-nm7dh\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.554232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.554224    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjkvw\" (UniqueName: \"kubernetes.io/projected/890bb418-26ac-4a5a-9abe-99de37cbccf3-kube-api-access-sjkvw\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.688839 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.688720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:00.792118 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:00.792070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:01.210753 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.210720    2580 generic.go:356] "Generic (PLEG): container finished" podID="de2d5069-c1f7-4362-a802-ece4efcfbf3b" containerID="153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae" exitCode=0
Sep 11 23:32:01.211223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.210767    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"
Sep 11 23:32:01.211223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.210803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx" event={"ID":"de2d5069-c1f7-4362-a802-ece4efcfbf3b","Type":"ContainerDied","Data":"153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae"}
Sep 11 23:32:01.211223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.210887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx" event={"ID":"de2d5069-c1f7-4362-a802-ece4efcfbf3b","Type":"ContainerDied","Data":"ede1a07280d9041830568e4fc2c8dd1478f0dfa4490e7f73374107e7bf8ffea0"}
Sep 11 23:32:01.211223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.210958    2580 scope.go:117] "RemoveContainer" containerID="153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae"
Sep 11 23:32:01.216221 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.216191    2580 generic.go:356] "Generic (PLEG): container finished" podID="890bb418-26ac-4a5a-9abe-99de37cbccf3" containerID="1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b" exitCode=0
Sep 11 23:32:01.216397 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.216234    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"
Sep 11 23:32:01.216397 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.216273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4" event={"ID":"890bb418-26ac-4a5a-9abe-99de37cbccf3","Type":"ContainerDied","Data":"1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b"}
Sep 11 23:32:01.216397 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.216331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4" event={"ID":"890bb418-26ac-4a5a-9abe-99de37cbccf3","Type":"ContainerDied","Data":"e95e29fd863b352d36429ee4bbca9aa01558e8a6cef858507293ad2b2bfbd01e"}
Sep 11 23:32:01.222570 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.222548    2580 scope.go:117] "RemoveContainer" containerID="153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae"
Sep 11 23:32:01.222880 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:01.222860    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae\": container with ID starting with 153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae not found: ID does not exist" containerID="153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae"
Sep 11 23:32:01.222973 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.222890    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae"} err="failed to get container status \"153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae\": rpc error: code = NotFound desc = could not find container \"153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae\": container with ID starting with 153af3d535296937fc5176ac70e03422a7f54819cfb9a8af50eb5aa94b9ea4ae not found: ID does not exist"
Sep 11 23:32:01.223023 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.222907    2580 scope.go:117] "RemoveContainer" containerID="1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b"
Sep 11 23:32:01.231713 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.231694    2580 scope.go:117] "RemoveContainer" containerID="1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b"
Sep 11 23:32:01.232054 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:01.232030    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b\": container with ID starting with 1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b not found: ID does not exist" containerID="1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b"
Sep 11 23:32:01.232150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.232061    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b"} err="failed to get container status \"1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b\": rpc error: code = NotFound desc = could not find container \"1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b\": container with ID starting with 1a0cc3a93d4f309afd8956a4e9f16dd2757ffa6f3d9b2d841b88bf04d833e04b not found: ID does not exist"
Sep 11 23:32:01.237337 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.237311    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"]
Sep 11 23:32:01.240994 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.240972    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-cd48eb-6dc9cf949d-zrwqx"]
Sep 11 23:32:01.252242 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.252214    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"]
Sep 11 23:32:01.257963 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.257938    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-cd48eb-56659ccdf-nj9f4"]
Sep 11 23:32:01.414134 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.414103    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="890bb418-26ac-4a5a-9abe-99de37cbccf3" path="/var/lib/kubelet/pods/890bb418-26ac-4a5a-9abe-99de37cbccf3/volumes"
Sep 11 23:32:01.414486 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.414473    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de2d5069-c1f7-4362-a802-ece4efcfbf3b" path="/var/lib/kubelet/pods/de2d5069-c1f7-4362-a802-ece4efcfbf3b/volumes"
Sep 11 23:32:01.567067 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:01.567018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:02.383370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:02.383332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:03.470465 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:03.470434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:04.278374 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:04.278337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:07.026223 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:07.026189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:07.127634 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:07.127570    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:07.305297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:07.305210    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:07.894629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:07.894597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:08.770551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:08.770517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:08.920533 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:08.920502    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:09.024386 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024305    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"]
Sep 11 23:32:09.024680 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024666    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890bb418-26ac-4a5a-9abe-99de37cbccf3" containerName="payload-processing"
Sep 11 23:32:09.024726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024682    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="890bb418-26ac-4a5a-9abe-99de37cbccf3" containerName="payload-processing"
Sep 11 23:32:09.024726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024692    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39975ea3-fa00-4ae0-adf0-b15e9821ccd4" containerName="payload-processing"
Sep 11 23:32:09.024726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024698    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="39975ea3-fa00-4ae0-adf0-b15e9821ccd4" containerName="payload-processing"
Sep 11 23:32:09.024726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024709    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" containerName="payload-processing"
Sep 11 23:32:09.024726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024714    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" containerName="payload-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024728    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="105e5c69-f920-4ccb-9659-140d0bf31b72" containerName="payload-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024734    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="105e5c69-f920-4ccb-9659-140d0bf31b72" containerName="payload-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024742    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2d5069-c1f7-4362-a802-ece4efcfbf3b" containerName="payload-pre-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024747    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2d5069-c1f7-4362-a802-ece4efcfbf3b" containerName="payload-pre-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024818    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="105e5c69-f920-4ccb-9659-140d0bf31b72" containerName="payload-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024827    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="890bb418-26ac-4a5a-9abe-99de37cbccf3" containerName="payload-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024832    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="de2d5069-c1f7-4362-a802-ece4efcfbf3b" containerName="payload-pre-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024838    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f5b0ddc-f4fc-4f4c-81f1-ce3bf6b453a0" containerName="payload-processing"
Sep 11 23:32:09.024873 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.024846    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="39975ea3-fa00-4ae0-adf0-b15e9821ccd4" containerName="payload-processing"
Sep 11 23:32:09.029472 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.029451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"
Sep 11 23:32:09.032836 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.032817    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 23:32:09.033154 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.033134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vrx7m\""
Sep 11 23:32:09.038090 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.038058    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"]
Sep 11 23:32:09.143898 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.143855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcd4f54-6eab-4ff7-803e-4d71eba14b8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p\" (UID: \"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"
Sep 11 23:32:09.244933 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.244877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcd4f54-6eab-4ff7-803e-4d71eba14b8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p\" (UID: \"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"
Sep 11 23:32:09.245588 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.245563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcd4f54-6eab-4ff7-803e-4d71eba14b8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p\" (UID: \"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"
Sep 11 23:32:09.340293 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.340258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"
Sep 11 23:32:09.611211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.611187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"]
Sep 11 23:32:09.613557 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:32:09.613522    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcd4f54_6eab_4ff7_803e_4d71eba14b8e.slice/crio-5f6f3629352fbae95d31c8b19198bd773d5add01ec29a8479594785cfc43ce0e WatchSource:0}: Error finding container 5f6f3629352fbae95d31c8b19198bd773d5add01ec29a8479594785cfc43ce0e: Status 404 returned error can't find the container with id 5f6f3629352fbae95d31c8b19198bd773d5add01ec29a8479594785cfc43ce0e
Sep 11 23:32:09.759581 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:09.759550    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:10.267629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:10.267541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p" event={"ID":"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e","Type":"ContainerStarted","Data":"615c8de6c16639ce36c145c433980799660bc07cd1b199eb2e24d921a5c9130b"}
Sep 11 23:32:10.267629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:10.267608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p" event={"ID":"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e","Type":"ContainerStarted","Data":"5f6f3629352fbae95d31c8b19198bd773d5add01ec29a8479594785cfc43ce0e"}
Sep 11 23:32:10.285932 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:10.285872    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p" podStartSLOduration=1.285856714 podStartE2EDuration="1.285856714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:32:10.284206563 +0000 UTC m=+1845.496177738" watchObservedRunningTime="2026-09-11 23:32:10.285856714 +0000 UTC m=+1845.497827888"
Sep 11 23:32:10.555171 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:10.555082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:11.275184 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:11.275147    2580 generic.go:356] "Generic (PLEG): container finished" podID="2fcd4f54-6eab-4ff7-803e-4d71eba14b8e" containerID="615c8de6c16639ce36c145c433980799660bc07cd1b199eb2e24d921a5c9130b" exitCode=0
Sep 11 23:32:11.275565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:11.275229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p" event={"ID":"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e","Type":"ContainerDied","Data":"615c8de6c16639ce36c145c433980799660bc07cd1b199eb2e24d921a5c9130b"}
Sep 11 23:32:11.309871 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:11.309826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:12.224064 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:12.224022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:12.409976 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:12.409950    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"
Sep 11 23:32:12.475767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:12.475678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcd4f54-6eab-4ff7-803e-4d71eba14b8e-maas-api-service-ca\") pod \"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e\" (UID: \"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e\") "
Sep 11 23:32:12.476119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:12.476099    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcd4f54-6eab-4ff7-803e-4d71eba14b8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2fcd4f54-6eab-4ff7-803e-4d71eba14b8e" (UID: "2fcd4f54-6eab-4ff7-803e-4d71eba14b8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:12.577449 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:12.577408    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcd4f54-6eab-4ff7-803e-4d71eba14b8e-maas-api-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:13.070750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:13.070709    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:13.287447 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:13.287413    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"
Sep 11 23:32:13.287624 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:13.287449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p" event={"ID":"2fcd4f54-6eab-4ff7-803e-4d71eba14b8e","Type":"ContainerDied","Data":"5f6f3629352fbae95d31c8b19198bd773d5add01ec29a8479594785cfc43ce0e"}
Sep 11 23:32:13.287624 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:13.287496    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f6f3629352fbae95d31c8b19198bd773d5add01ec29a8479594785cfc43ce0e"
Sep 11 23:32:13.896263 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:13.896223    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"]
Sep 11 23:32:13.991182 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:13.991150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:17.741318 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:17.741268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:18.361267 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.361229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:18.406718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.406683    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"]
Sep 11 23:32:18.407015 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.406986    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x" podUID="21a01628-25d7-44bc-9096-8e1e3ee5aa7b" containerName="payload-processing" containerID="cri-o://2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68" gracePeriod=30
Sep 11 23:32:18.416802 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.416775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"]
Sep 11 23:32:18.417042 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.417013    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6" podUID="80159859-d10a-4267-b5cc-05ceef02a4bc" containerName="payload-pre-processing" containerID="cri-o://ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5" gracePeriod=30
Sep 11 23:32:18.453961 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.453928    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:18.523999 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.523966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:18.601368 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.601338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:18.662685 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.662659    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:32:18.676737 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.676708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:18.697942 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.697901    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:32:18.731422 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.731383    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xx4mx\" (UniqueName: \"kubernetes.io/projected/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-kube-api-access-xx4mx\") pod \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") "
Sep 11 23:32:18.731610 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.731464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-plugins-config-volume\") pod \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\" (UID: \"21a01628-25d7-44bc-9096-8e1e3ee5aa7b\") "
Sep 11 23:32:18.731925 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.731882    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "21a01628-25d7-44bc-9096-8e1e3ee5aa7b" (UID: "21a01628-25d7-44bc-9096-8e1e3ee5aa7b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:18.733513 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.733489    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-kube-api-access-xx4mx" (OuterVolumeSpecName: "kube-api-access-xx4mx") pod "21a01628-25d7-44bc-9096-8e1e3ee5aa7b" (UID: "21a01628-25d7-44bc-9096-8e1e3ee5aa7b"). InnerVolumeSpecName "kube-api-access-xx4mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:18.832031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.831993    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhxxd\" (UniqueName: \"kubernetes.io/projected/80159859-d10a-4267-b5cc-05ceef02a4bc-kube-api-access-hhxxd\") pod \"80159859-d10a-4267-b5cc-05ceef02a4bc\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") "
Sep 11 23:32:18.832031 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.832041    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159859-d10a-4267-b5cc-05ceef02a4bc-plugins-config-volume\") pod \"80159859-d10a-4267-b5cc-05ceef02a4bc\" (UID: \"80159859-d10a-4267-b5cc-05ceef02a4bc\") "
Sep 11 23:32:18.832432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.832263    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx4mx\" (UniqueName: \"kubernetes.io/projected/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-kube-api-access-xx4mx\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.832432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.832275    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21a01628-25d7-44bc-9096-8e1e3ee5aa7b-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.832506 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.832423    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80159859-d10a-4267-b5cc-05ceef02a4bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80159859-d10a-4267-b5cc-05ceef02a4bc" (UID: "80159859-d10a-4267-b5cc-05ceef02a4bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:18.834090 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.834068    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80159859-d10a-4267-b5cc-05ceef02a4bc-kube-api-access-hhxxd" (OuterVolumeSpecName: "kube-api-access-hhxxd") pod "80159859-d10a-4267-b5cc-05ceef02a4bc" (UID: "80159859-d10a-4267-b5cc-05ceef02a4bc"). InnerVolumeSpecName "kube-api-access-hhxxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:18.932968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.932848    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhxxd\" (UniqueName: \"kubernetes.io/projected/80159859-d10a-4267-b5cc-05ceef02a4bc-kube-api-access-hhxxd\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.932968 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.932883    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80159859-d10a-4267-b5cc-05ceef02a4bc-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.942156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:18.942125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:19.085354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.085312    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:19.318903 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.318866    2580 generic.go:356] "Generic (PLEG): container finished" podID="21a01628-25d7-44bc-9096-8e1e3ee5aa7b" containerID="2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68" exitCode=0
Sep 11 23:32:19.319107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.318906    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"
Sep 11 23:32:19.319107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.318905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x" event={"ID":"21a01628-25d7-44bc-9096-8e1e3ee5aa7b","Type":"ContainerDied","Data":"2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68"}
Sep 11 23:32:19.319107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.318986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x" event={"ID":"21a01628-25d7-44bc-9096-8e1e3ee5aa7b","Type":"ContainerDied","Data":"7c8b9ebf89ba4676632fa13dc19d299c65656fd61cd39e8a8ee93f8bdbab205d"}
Sep 11 23:32:19.319107 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.319020    2580 scope.go:117] "RemoveContainer" containerID="2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68"
Sep 11 23:32:19.320442 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.320419    2580 generic.go:356] "Generic (PLEG): container finished" podID="80159859-d10a-4267-b5cc-05ceef02a4bc" containerID="ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5" exitCode=0
Sep 11 23:32:19.320536 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.320450    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"
Sep 11 23:32:19.320536 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.320474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6" event={"ID":"80159859-d10a-4267-b5cc-05ceef02a4bc","Type":"ContainerDied","Data":"ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5"}
Sep 11 23:32:19.320536 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.320513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6" event={"ID":"80159859-d10a-4267-b5cc-05ceef02a4bc","Type":"ContainerDied","Data":"7116f85322d8391c35ef387a9dd752cc3a164839a1bfbdc56759cc86ed5333b2"}
Sep 11 23:32:19.328027 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.327994    2580 scope.go:117] "RemoveContainer" containerID="2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68"
Sep 11 23:32:19.328369 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:19.328348    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68\": container with ID starting with 2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68 not found: ID does not exist" containerID="2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68"
Sep 11 23:32:19.328459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.328376    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68"} err="failed to get container status \"2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68\": rpc error: code = NotFound desc = could not find container \"2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68\": container with ID starting with 2f3aee1675cd5f8dc4e61cd8fe6071186b75a9137a240d7cedb181ed73d91f68 not found: ID does not exist"
Sep 11 23:32:19.328459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.328399    2580 scope.go:117] "RemoveContainer" containerID="ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5"
Sep 11 23:32:19.336765 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.336748    2580 scope.go:117] "RemoveContainer" containerID="ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5"
Sep 11 23:32:19.337152 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:19.337131    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5\": container with ID starting with ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5 not found: ID does not exist" containerID="ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5"
Sep 11 23:32:19.337231 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.337160    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5"} err="failed to get container status \"ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5\": rpc error: code = NotFound desc = could not find container \"ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5\": container with ID starting with ce3e5f4a7b93bbc63604d07ae052bbb7c6685c4f0447ab81202b1498b8a456b5 not found: ID does not exist"
Sep 11 23:32:19.346365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.346334    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"]
Sep 11 23:32:19.352096 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.352071    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-97d9dc-85575b58c-sn89x"]
Sep 11 23:32:19.363421 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.363394    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"]
Sep 11 23:32:19.368105 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.368082    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-97d9dc-56cd4d986b-fx5n6"]
Sep 11 23:32:19.413840 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.413810    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21a01628-25d7-44bc-9096-8e1e3ee5aa7b" path="/var/lib/kubelet/pods/21a01628-25d7-44bc-9096-8e1e3ee5aa7b/volumes"
Sep 11 23:32:19.414211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:19.414195    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80159859-d10a-4267-b5cc-05ceef02a4bc" path="/var/lib/kubelet/pods/80159859-d10a-4267-b5cc-05ceef02a4bc/volumes"
Sep 11 23:32:20.075320 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:20.075281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:20.970544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:20.970508    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:24.799366 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:24.799333    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"]
Sep 11 23:32:24.802875 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:24.802848    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-06ef79-29819490-w56l8"]
Sep 11 23:32:24.803368 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:24.803344    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:24.828091 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:24.828055    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"]
Sep 11 23:32:24.828359 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:24.828330    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252" podUID="34bf8857-aeb6-474d-9d58-52bf66df9534" containerName="payload-processing" containerID="cri-o://40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7" gracePeriod=30
Sep 11 23:32:24.881804 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:24.881775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:24.982580 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:24.982547    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:25.072768 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.072735    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"]
Sep 11 23:32:25.080579 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.080550    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-06ef79-wfl4p"]
Sep 11 23:32:25.084480 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.084461    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:32:25.091878 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.091856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:25.188240 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.188206    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gm8t\" (UniqueName: \"kubernetes.io/projected/34bf8857-aeb6-474d-9d58-52bf66df9534-kube-api-access-7gm8t\") pod \"34bf8857-aeb6-474d-9d58-52bf66df9534\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") "
Sep 11 23:32:25.188409 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.188311    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bf8857-aeb6-474d-9d58-52bf66df9534-plugins-config-volume\") pod \"34bf8857-aeb6-474d-9d58-52bf66df9534\" (UID: \"34bf8857-aeb6-474d-9d58-52bf66df9534\") "
Sep 11 23:32:25.188698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.188668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34bf8857-aeb6-474d-9d58-52bf66df9534-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34bf8857-aeb6-474d-9d58-52bf66df9534" (UID: "34bf8857-aeb6-474d-9d58-52bf66df9534"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:25.190416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.190385    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34bf8857-aeb6-474d-9d58-52bf66df9534-kube-api-access-7gm8t" (OuterVolumeSpecName: "kube-api-access-7gm8t") pod "34bf8857-aeb6-474d-9d58-52bf66df9534" (UID: "34bf8857-aeb6-474d-9d58-52bf66df9534"). InnerVolumeSpecName "kube-api-access-7gm8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:25.289183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.289147    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gm8t\" (UniqueName: \"kubernetes.io/projected/34bf8857-aeb6-474d-9d58-52bf66df9534-kube-api-access-7gm8t\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:25.289183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.289179    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bf8857-aeb6-474d-9d58-52bf66df9534-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:25.353288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.353191    2580 generic.go:356] "Generic (PLEG): container finished" podID="34bf8857-aeb6-474d-9d58-52bf66df9534" containerID="40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7" exitCode=0
Sep 11 23:32:25.353288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.353237    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"
Sep 11 23:32:25.353288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.353274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252" event={"ID":"34bf8857-aeb6-474d-9d58-52bf66df9534","Type":"ContainerDied","Data":"40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7"}
Sep 11 23:32:25.353535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.353326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252" event={"ID":"34bf8857-aeb6-474d-9d58-52bf66df9534","Type":"ContainerDied","Data":"8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c"}
Sep 11 23:32:25.353535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.353350    2580 scope.go:117] "RemoveContainer" containerID="40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7"
Sep 11 23:32:25.362618 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.362594    2580 scope.go:117] "RemoveContainer" containerID="40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7"
Sep 11 23:32:25.362994 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:25.362968    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7\": container with ID starting with 40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7 not found: ID does not exist" containerID="40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7"
Sep 11 23:32:25.363082 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.363007    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7"} err="failed to get container status \"40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7\": rpc error: code = NotFound desc = could not find container \"40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7\": container with ID starting with 40b3f751f8ed54d90a36d06c014c32c049204d2dc8ce27233cec5254853d51f7 not found: ID does not exist"
Sep 11 23:32:25.389483 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.389452    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"]
Sep 11 23:32:25.399564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.399537    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-06ef79-6886bf78c7-pn252"]
Sep 11 23:32:25.413815 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.413783    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fcd4f54-6eab-4ff7-803e-4d71eba14b8e" path="/var/lib/kubelet/pods/2fcd4f54-6eab-4ff7-803e-4d71eba14b8e/volumes"
Sep 11 23:32:25.414254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.414233    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34bf8857-aeb6-474d-9d58-52bf66df9534" path="/var/lib/kubelet/pods/34bf8857-aeb6-474d-9d58-52bf66df9534/volumes"
Sep 11 23:32:25.414544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.414527    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37" path="/var/lib/kubelet/pods/57a7aacc-5ed4-4e5d-aed0-7dd3b0aabb37/volumes"
Sep 11 23:32:25.505438 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.505405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:25.630638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.630537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:25.788275 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.788235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:25.870458 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:25.870426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:26.058742 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:26.058651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:26.672500 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:26.672469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:27.159942 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:27.159897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:27.264433 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:27.264402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:27.345298 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:27.345258    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:32:27.461481 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:27.461391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:28.915565 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:28.915523    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x" podUID="3845c9a0-7ffe-479b-a088-9b60f62f8c66" containerName="main" containerID="cri-o://3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633" gracePeriod=45
Sep 11 23:32:29.171411 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.171337    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:32:29.327164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327128    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/3845c9a0-7ffe-479b-a088-9b60f62f8c66-kube-api-access-4j42k\") pod \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") "
Sep 11 23:32:29.327164 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327165    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tls-certs\") pod \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") "
Sep 11 23:32:29.327416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-dshm\") pod \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") "
Sep 11 23:32:29.327416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327237    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-model-cache\") pod \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") "
Sep 11 23:32:29.327416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327260    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tmp-dir\") pod \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") "
Sep 11 23:32:29.327416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-home\") pod \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\" (UID: \"3845c9a0-7ffe-479b-a088-9b60f62f8c66\") "
Sep 11 23:32:29.327643 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327609    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-model-cache" (OuterVolumeSpecName: "model-cache") pod "3845c9a0-7ffe-479b-a088-9b60f62f8c66" (UID: "3845c9a0-7ffe-479b-a088-9b60f62f8c66"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.327727 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327696    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3845c9a0-7ffe-479b-a088-9b60f62f8c66" (UID: "3845c9a0-7ffe-479b-a088-9b60f62f8c66"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.327970 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.327949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-home" (OuterVolumeSpecName: "home") pod "3845c9a0-7ffe-479b-a088-9b60f62f8c66" (UID: "3845c9a0-7ffe-479b-a088-9b60f62f8c66"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.329474 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.329443    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-dshm" (OuterVolumeSpecName: "dshm") pod "3845c9a0-7ffe-479b-a088-9b60f62f8c66" (UID: "3845c9a0-7ffe-479b-a088-9b60f62f8c66"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.329546 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.329478    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3845c9a0-7ffe-479b-a088-9b60f62f8c66" (UID: "3845c9a0-7ffe-479b-a088-9b60f62f8c66"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:29.329706 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.329680    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3845c9a0-7ffe-479b-a088-9b60f62f8c66-kube-api-access-4j42k" (OuterVolumeSpecName: "kube-api-access-4j42k") pod "3845c9a0-7ffe-479b-a088-9b60f62f8c66" (UID: "3845c9a0-7ffe-479b-a088-9b60f62f8c66"). InnerVolumeSpecName "kube-api-access-4j42k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:29.373319 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.373284    2580 generic.go:356] "Generic (PLEG): container finished" podID="3845c9a0-7ffe-479b-a088-9b60f62f8c66" containerID="3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633" exitCode=0
Sep 11 23:32:29.373496 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.373326    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"
Sep 11 23:32:29.373496 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.373351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x" event={"ID":"3845c9a0-7ffe-479b-a088-9b60f62f8c66","Type":"ContainerDied","Data":"3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633"}
Sep 11 23:32:29.373496 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.373386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x" event={"ID":"3845c9a0-7ffe-479b-a088-9b60f62f8c66","Type":"ContainerDied","Data":"445e394b96fadb3e5191ebc7b2fbfc1a0a10d8afd76e3580a901c3ddd4f989ff"}
Sep 11 23:32:29.373496 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.373402    2580 scope.go:117] "RemoveContainer" containerID="3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633"
Sep 11 23:32:29.382609 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.382589    2580 scope.go:117] "RemoveContainer" containerID="3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633"
Sep 11 23:32:29.382893 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:29.382870    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633\": container with ID starting with 3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633 not found: ID does not exist" containerID="3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633"
Sep 11 23:32:29.382969 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.382904    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633"} err="failed to get container status \"3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633\": rpc error: code = NotFound desc = could not find container \"3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633\": container with ID starting with 3e53b691d0ec1b91e34f5db57dad8d1f84b3b7869c60cf6f5ee0c41070221633 not found: ID does not exist"
Sep 11 23:32:29.396365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.396330    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"]
Sep 11 23:32:29.402119 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.402092    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-w4-06ef79-kserve-759c67dc4b-8kp2x"]
Sep 11 23:32:29.413416 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.413382    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3845c9a0-7ffe-479b-a088-9b60f62f8c66" path="/var/lib/kubelet/pods/3845c9a0-7ffe-479b-a088-9b60f62f8c66/volumes"
Sep 11 23:32:29.428269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.428202    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/3845c9a0-7ffe-479b-a088-9b60f62f8c66-kube-api-access-4j42k\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.428269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.428230    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.428269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.428244    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.428269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.428252    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.428269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.428260    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.428269 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:29.428268    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3845c9a0-7ffe-479b-a088-9b60f62f8c66-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:30.465551 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:30.465506    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache]"
Sep 11 23:32:31.559140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:31.559093    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:31.659792 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:31.659747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:31.823354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:31.823323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:40.507412 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:40.507369    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:32:42.302346 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:42.302306    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache]"
Sep 11 23:32:43.963134 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:43.963097    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"]
Sep 11 23:32:43.989965 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:43.989936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:44.916043 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:44.916012    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"]
Sep 11 23:32:44.940670 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:44.940638    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:50.555468 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:50.555379    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:32:51.278831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:51.278794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:51.359882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:51.359846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:51.960924 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:51.960874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:32:57.353425 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:57.353385    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:32:58.981643 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:58.981593    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z" podUID="07635a6a-b0af-422f-9eaa-c85c96ebcbdb" containerName="main" containerID="cri-o://3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1" gracePeriod=45
Sep 11 23:32:59.223396 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.223368    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:32:59.292058 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.291961    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-dshm\") pod \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") "
Sep 11 23:32:59.292058 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292025    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-model-cache\") pod \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") "
Sep 11 23:32:59.292292 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292072    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tls-certs\") pod \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") "
Sep 11 23:32:59.292292 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292116    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdf9r\" (UniqueName: \"kubernetes.io/projected/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-kube-api-access-sdf9r\") pod \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") "
Sep 11 23:32:59.292292 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292142    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tmp-dir\") pod \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") "
Sep 11 23:32:59.292292 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-home\") pod \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\" (UID: \"07635a6a-b0af-422f-9eaa-c85c96ebcbdb\") "
Sep 11 23:32:59.292493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292351    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-model-cache" (OuterVolumeSpecName: "model-cache") pod "07635a6a-b0af-422f-9eaa-c85c96ebcbdb" (UID: "07635a6a-b0af-422f-9eaa-c85c96ebcbdb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:59.292493 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292448    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07635a6a-b0af-422f-9eaa-c85c96ebcbdb" (UID: "07635a6a-b0af-422f-9eaa-c85c96ebcbdb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:59.292606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.292489    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-home" (OuterVolumeSpecName: "home") pod "07635a6a-b0af-422f-9eaa-c85c96ebcbdb" (UID: "07635a6a-b0af-422f-9eaa-c85c96ebcbdb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:59.294323 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.294296    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-dshm" (OuterVolumeSpecName: "dshm") pod "07635a6a-b0af-422f-9eaa-c85c96ebcbdb" (UID: "07635a6a-b0af-422f-9eaa-c85c96ebcbdb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:59.294417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.294318    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07635a6a-b0af-422f-9eaa-c85c96ebcbdb" (UID: "07635a6a-b0af-422f-9eaa-c85c96ebcbdb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:59.294417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.294299    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-kube-api-access-sdf9r" (OuterVolumeSpecName: "kube-api-access-sdf9r") pod "07635a6a-b0af-422f-9eaa-c85c96ebcbdb" (UID: "07635a6a-b0af-422f-9eaa-c85c96ebcbdb"). InnerVolumeSpecName "kube-api-access-sdf9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:59.393080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.393043    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:59.393080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.393075    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:59.393080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.393085    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:59.393080 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.393094    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdf9r\" (UniqueName: \"kubernetes.io/projected/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-kube-api-access-sdf9r\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:59.393365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.393124    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:59.393365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.393133    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07635a6a-b0af-422f-9eaa-c85c96ebcbdb-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:32:59.517429 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.517396    2580 generic.go:356] "Generic (PLEG): container finished" podID="07635a6a-b0af-422f-9eaa-c85c96ebcbdb" containerID="3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1" exitCode=0
Sep 11 23:32:59.517616 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.517433    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"
Sep 11 23:32:59.517616 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.517478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z" event={"ID":"07635a6a-b0af-422f-9eaa-c85c96ebcbdb","Type":"ContainerDied","Data":"3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1"}
Sep 11 23:32:59.517616 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.517520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z" event={"ID":"07635a6a-b0af-422f-9eaa-c85c96ebcbdb","Type":"ContainerDied","Data":"57acfe16b917ad51f5f81762f43dac98cfaeeeffca75d38bfe93342256623609"}
Sep 11 23:32:59.517616 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.517537    2580 scope.go:117] "RemoveContainer" containerID="3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1"
Sep 11 23:32:59.525710 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.525687    2580 scope.go:117] "RemoveContainer" containerID="3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1"
Sep 11 23:32:59.526012 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:32:59.525990    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1\": container with ID starting with 3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1 not found: ID does not exist" containerID="3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1"
Sep 11 23:32:59.526085 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.526024    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1"} err="failed to get container status \"3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1\": rpc error: code = NotFound desc = could not find container \"3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1\": container with ID starting with 3d4b615fae25d5c79e6b5fc2fe3f79f6e270a7b1c80a8ac163812dc47095daf1 not found: ID does not exist"
Sep 11 23:32:59.538812 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.538790    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"]
Sep 11 23:32:59.542469 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.542417    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/auth-test-model-1c4628-kserve-744bd586f5-tdp4z"]
Sep 11 23:32:59.934777 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:32:59.934700    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd" podUID="a2be8496-11c2-4253-9f68-5f05c36211fd" containerName="main" containerID="cri-o://ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c" gracePeriod=45
Sep 11 23:33:00.180005 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.179979    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:33:00.300239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300148    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-tmp-dir\") pod \"a2be8496-11c2-4253-9f68-5f05c36211fd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") "
Sep 11 23:33:00.300239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-home\") pod \"a2be8496-11c2-4253-9f68-5f05c36211fd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") "
Sep 11 23:33:00.300239 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300232    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-dshm\") pod \"a2be8496-11c2-4253-9f68-5f05c36211fd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") "
Sep 11 23:33:00.300481 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300253    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2be8496-11c2-4253-9f68-5f05c36211fd-tls-certs\") pod \"a2be8496-11c2-4253-9f68-5f05c36211fd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") "
Sep 11 23:33:00.300481 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300303    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz7tr\" (UniqueName: \"kubernetes.io/projected/a2be8496-11c2-4253-9f68-5f05c36211fd-kube-api-access-vz7tr\") pod \"a2be8496-11c2-4253-9f68-5f05c36211fd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") "
Sep 11 23:33:00.300481 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300354    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-model-cache\") pod \"a2be8496-11c2-4253-9f68-5f05c36211fd\" (UID: \"a2be8496-11c2-4253-9f68-5f05c36211fd\") "
Sep 11 23:33:00.300657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-home" (OuterVolumeSpecName: "home") pod "a2be8496-11c2-4253-9f68-5f05c36211fd" (UID: "a2be8496-11c2-4253-9f68-5f05c36211fd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:33:00.300657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300505    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2be8496-11c2-4253-9f68-5f05c36211fd" (UID: "a2be8496-11c2-4253-9f68-5f05c36211fd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:33:00.300657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300605    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-home\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:33:00.300814 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.300734    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2be8496-11c2-4253-9f68-5f05c36211fd" (UID: "a2be8496-11c2-4253-9f68-5f05c36211fd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:33:00.302598 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.302570    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-dshm" (OuterVolumeSpecName: "dshm") pod "a2be8496-11c2-4253-9f68-5f05c36211fd" (UID: "a2be8496-11c2-4253-9f68-5f05c36211fd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:33:00.302707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.302635    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2be8496-11c2-4253-9f68-5f05c36211fd-kube-api-access-vz7tr" (OuterVolumeSpecName: "kube-api-access-vz7tr") pod "a2be8496-11c2-4253-9f68-5f05c36211fd" (UID: "a2be8496-11c2-4253-9f68-5f05c36211fd"). InnerVolumeSpecName "kube-api-access-vz7tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:33:00.302707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.302687    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2be8496-11c2-4253-9f68-5f05c36211fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2be8496-11c2-4253-9f68-5f05c36211fd" (UID: "a2be8496-11c2-4253-9f68-5f05c36211fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:33:00.401454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.401410    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-model-cache\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:33:00.401454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.401448    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-tmp-dir\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:33:00.401454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.401460    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2be8496-11c2-4253-9f68-5f05c36211fd-dshm\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:33:00.401698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.401470    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2be8496-11c2-4253-9f68-5f05c36211fd-tls-certs\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:33:00.401698 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.401482    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz7tr\" (UniqueName: \"kubernetes.io/projected/a2be8496-11c2-4253-9f68-5f05c36211fd-kube-api-access-vz7tr\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:33:00.524870 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.524834    2580 generic.go:356] "Generic (PLEG): container finished" podID="a2be8496-11c2-4253-9f68-5f05c36211fd" containerID="ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c" exitCode=0
Sep 11 23:33:00.525133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.524880    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"
Sep 11 23:33:00.525133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.524949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd" event={"ID":"a2be8496-11c2-4253-9f68-5f05c36211fd","Type":"ContainerDied","Data":"ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c"}
Sep 11 23:33:00.525133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.524996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd" event={"ID":"a2be8496-11c2-4253-9f68-5f05c36211fd","Type":"ContainerDied","Data":"405d922be09bb1999d2059e5ad74567885639a52a372e219a20177296b7bccaa"}
Sep 11 23:33:00.525133 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.525019    2580 scope.go:117] "RemoveContainer" containerID="ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c"
Sep 11 23:33:00.534081 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.534063    2580 scope.go:117] "RemoveContainer" containerID="ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c"
Sep 11 23:33:00.534381 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:33:00.534361    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c\": container with ID starting with ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c not found: ID does not exist" containerID="ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c"
Sep 11 23:33:00.534437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.534389    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c"} err="failed to get container status \"ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c\": rpc error: code = NotFound desc = could not find container \"ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c\": container with ID starting with ab2a413c2409bb3a4defb05f286246c87761accf5f958546f55975d20128c50c not found: ID does not exist"
Sep 11 23:33:00.547632 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.547605    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"]
Sep 11 23:33:00.553420 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:00.553351    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-da9451/auth-test-model-da9451-kserve-df85cd7f5-695nd"]
Sep 11 23:33:00.564509 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:33:00.564476    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:33:01.414725 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:01.413981    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07635a6a-b0af-422f-9eaa-c85c96ebcbdb" path="/var/lib/kubelet/pods/07635a6a-b0af-422f-9eaa-c85c96ebcbdb/volumes"
Sep 11 23:33:01.414725 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:01.414606    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2be8496-11c2-4253-9f68-5f05c36211fd" path="/var/lib/kubelet/pods/a2be8496-11c2-4253-9f68-5f05c36211fd/volumes"
Sep 11 23:33:03.860987 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:33:03.860940    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache]"
Sep 11 23:33:03.861470 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:33:03.860940    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:33:07.391657 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:07.391616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:33:10.606643 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:33:10.606599    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache]"
Sep 11 23:33:11.628406 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:11.628373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:33:12.301574 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:33:12.301529    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:33:16.223379 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:16.223344    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:33:20.647884 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:33:20.647845    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice/crio-8951eb6011c1c7e1b4815ed51940a2b937793cc3a538eb614977e6f37ffc711c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf8857_aeb6_474d_9d58_52bf66df9534.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 23:33:45.276495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:33:45.276457    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:34:15.758996 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.758954    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7607-7aa66b"]
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759425    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80159859-d10a-4267-b5cc-05ceef02a4bc" containerName="payload-pre-processing"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759444    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="80159859-d10a-4267-b5cc-05ceef02a4bc" containerName="payload-pre-processing"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759461    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2be8496-11c2-4253-9f68-5f05c36211fd" containerName="main"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759470    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2be8496-11c2-4253-9f68-5f05c36211fd" containerName="main"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759484    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07635a6a-b0af-422f-9eaa-c85c96ebcbdb" containerName="main"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759493    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07635a6a-b0af-422f-9eaa-c85c96ebcbdb" containerName="main"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759502    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fcd4f54-6eab-4ff7-803e-4d71eba14b8e" containerName="revoke-keys"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759511    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fcd4f54-6eab-4ff7-803e-4d71eba14b8e" containerName="revoke-keys"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759529    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3845c9a0-7ffe-479b-a088-9b60f62f8c66" containerName="main"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759537    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3845c9a0-7ffe-479b-a088-9b60f62f8c66" containerName="main"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759547    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21a01628-25d7-44bc-9096-8e1e3ee5aa7b" containerName="payload-processing"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759555    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21a01628-25d7-44bc-9096-8e1e3ee5aa7b" containerName="payload-processing"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759574    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34bf8857-aeb6-474d-9d58-52bf66df9534" containerName="payload-processing"
Sep 11 23:34:15.759576 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759582    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="34bf8857-aeb6-474d-9d58-52bf66df9534" containerName="payload-processing"
Sep 11 23:34:15.760324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759666    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2be8496-11c2-4253-9f68-5f05c36211fd" containerName="main"
Sep 11 23:34:15.760324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759680    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="21a01628-25d7-44bc-9096-8e1e3ee5aa7b" containerName="payload-processing"
Sep 11 23:34:15.760324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759687    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3845c9a0-7ffe-479b-a088-9b60f62f8c66" containerName="main"
Sep 11 23:34:15.760324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759697    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07635a6a-b0af-422f-9eaa-c85c96ebcbdb" containerName="main"
Sep 11 23:34:15.760324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759706    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fcd4f54-6eab-4ff7-803e-4d71eba14b8e" containerName="revoke-keys"
Sep 11 23:34:15.760324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759715    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="34bf8857-aeb6-474d-9d58-52bf66df9534" containerName="payload-processing"
Sep 11 23:34:15.760324 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.759722    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="80159859-d10a-4267-b5cc-05ceef02a4bc" containerName="payload-pre-processing"
Sep 11 23:34:15.762930 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.762896    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:15.769862 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.769843    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8cxx9\""
Sep 11 23:34:15.776673 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.776649    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7607-7aa66b"]
Sep 11 23:34:15.849197 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.849156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcd9b\" (UniqueName: \"kubernetes.io/projected/2e8ad813-48e8-44eb-bd49-0870cd1011a5-kube-api-access-zcd9b\") pod \"test-curl-7607-7aa66b\" (UID: \"2e8ad813-48e8-44eb-bd49-0870cd1011a5\") " pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:15.949929 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.949887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcd9b\" (UniqueName: \"kubernetes.io/projected/2e8ad813-48e8-44eb-bd49-0870cd1011a5-kube-api-access-zcd9b\") pod \"test-curl-7607-7aa66b\" (UID: \"2e8ad813-48e8-44eb-bd49-0870cd1011a5\") " pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:15.959300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:15.959264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcd9b\" (UniqueName: \"kubernetes.io/projected/2e8ad813-48e8-44eb-bd49-0870cd1011a5-kube-api-access-zcd9b\") pod \"test-curl-7607-7aa66b\" (UID: \"2e8ad813-48e8-44eb-bd49-0870cd1011a5\") " pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:16.071341 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:16.071314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:16.343696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:16.343552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7607-7aa66b"]
Sep 11 23:34:16.346486 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:34:16.346444    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e8ad813_48e8_44eb_bd49_0870cd1011a5.slice/crio-c0ef625e4a44022c6fdcf2830501324a0f68b84bdfd02a598399bd63478552af WatchSource:0}: Error finding container c0ef625e4a44022c6fdcf2830501324a0f68b84bdfd02a598399bd63478552af: Status 404 returned error can't find the container with id c0ef625e4a44022c6fdcf2830501324a0f68b84bdfd02a598399bd63478552af
Sep 11 23:34:16.883017 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:16.882982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7607-7aa66b" event={"ID":"2e8ad813-48e8-44eb-bd49-0870cd1011a5","Type":"ContainerStarted","Data":"c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604"}
Sep 11 23:34:16.883017 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:16.883019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7607-7aa66b" event={"ID":"2e8ad813-48e8-44eb-bd49-0870cd1011a5","Type":"ContainerStarted","Data":"c0ef625e4a44022c6fdcf2830501324a0f68b84bdfd02a598399bd63478552af"}
Sep 11 23:34:16.897300 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:16.897244    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7607-7aa66b" podStartSLOduration=1.897228701 podStartE2EDuration="1.897228701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:34:16.896406884 +0000 UTC m=+1972.108378060" watchObservedRunningTime="2026-09-11 23:34:16.897228701 +0000 UTC m=+1972.109199874"
Sep 11 23:34:17.526738 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:17.526697    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7607-7aa66b"]
Sep 11 23:34:17.530991 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:17.530892    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7607-7aa66b"]
Sep 11 23:34:17.887472 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:34:17.887440    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7607-7aa66b" container="test-curl-7607-7aa66b"
Sep 11 23:34:18.891676 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:18.891628    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7607-7aa66b" podUID="2e8ad813-48e8-44eb-bd49-0870cd1011a5" containerName="test-curl-7607-7aa66b" containerID="cri-o://c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604" gracePeriod=2
Sep 11 23:34:18.896270 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:34:18.896243    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7607-7aa66b\" is forbidden: User \"system:node:ip-10-0-138-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-44.ec2.internal' and this object" podUID="2e8ad813-48e8-44eb-bd49-0870cd1011a5" pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:21.081202 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.081178    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:21.088233 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.088210    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcd9b\" (UniqueName: \"kubernetes.io/projected/2e8ad813-48e8-44eb-bd49-0870cd1011a5-kube-api-access-zcd9b\") pod \"2e8ad813-48e8-44eb-bd49-0870cd1011a5\" (UID: \"2e8ad813-48e8-44eb-bd49-0870cd1011a5\") "
Sep 11 23:34:21.090181 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.090154    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e8ad813-48e8-44eb-bd49-0870cd1011a5-kube-api-access-zcd9b" (OuterVolumeSpecName: "kube-api-access-zcd9b") pod "2e8ad813-48e8-44eb-bd49-0870cd1011a5" (UID: "2e8ad813-48e8-44eb-bd49-0870cd1011a5"). InnerVolumeSpecName "kube-api-access-zcd9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:34:21.159331 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.159222    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:34:21.189544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.189509    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcd9b\" (UniqueName: \"kubernetes.io/projected/2e8ad813-48e8-44eb-bd49-0870cd1011a5-kube-api-access-zcd9b\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:34:21.413533 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.413451    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e8ad813-48e8-44eb-bd49-0870cd1011a5" path="/var/lib/kubelet/pods/2e8ad813-48e8-44eb-bd49-0870cd1011a5/volumes"
Sep 11 23:34:21.906665 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.906633    2580 generic.go:356] "Generic (PLEG): container finished" podID="2e8ad813-48e8-44eb-bd49-0870cd1011a5" containerID="c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604" exitCode=137
Sep 11 23:34:21.906834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.906669    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-7aa66b"
Sep 11 23:34:21.906834 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.906698    2580 scope.go:117] "RemoveContainer" containerID="c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604"
Sep 11 23:34:21.914786 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.914770    2580 scope.go:117] "RemoveContainer" containerID="c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604"
Sep 11 23:34:21.915106 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:34:21.915086    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604\": container with ID starting with c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604 not found: ID does not exist" containerID="c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604"
Sep 11 23:34:21.915184 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:21.915118    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604"} err="failed to get container status \"c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604\": rpc error: code = NotFound desc = could not find container \"c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604\": container with ID starting with c815d84c2ee498497912c3dc75300c1355f7e9d548101034cfbf32d277025604 not found: ID does not exist"
Sep 11 23:34:22.081904 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:22.081869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:34:33.843886 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:34:33.843827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:35:09.104123 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:35:09.104082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:35:14.881536 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:35:14.881496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:35:47.725109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:35:47.724989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:35:49.092736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:35:49.092700    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:03.436056 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:03.436021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:04.351647 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:04.351593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:04.769622 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:04.769516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:05.198206 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:05.198165    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:05.401482 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:05.401450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:05.824396 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:05.824362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:08.272157 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:08.272121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:08.377771 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:08.377722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:08.476404 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:08.476363    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:09.303480 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:09.303444    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:10.280877 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.280838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:10.380450 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.380413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:10.479122 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.479087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:10.774539 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.774508    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:10.867242 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.867211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:10.899522 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.899488    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"]
Sep 11 23:36:10.899829 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.899816    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e8ad813-48e8-44eb-bd49-0870cd1011a5" containerName="test-curl-7607-7aa66b"
Sep 11 23:36:10.899874 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.899831    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e8ad813-48e8-44eb-bd49-0870cd1011a5" containerName="test-curl-7607-7aa66b"
Sep 11 23:36:10.899908 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.899886    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e8ad813-48e8-44eb-bd49-0870cd1011a5" containerName="test-curl-7607-7aa66b"
Sep 11 23:36:10.902724 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.902708    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:10.906001 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.905980    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-1fb90b\""
Sep 11 23:36:10.915047 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.915024    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"]
Sep 11 23:36:10.937185 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.937151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxrx9\" (UniqueName: \"kubernetes.io/projected/7a2ad887-2bf1-4008-a61a-f7092ba4885d-kube-api-access-qxrx9\") pod \"payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:10.937360 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:10.937236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2ad887-2bf1-4008-a61a-f7092ba4885d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:11.038184 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:11.038097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2ad887-2bf1-4008-a61a-f7092ba4885d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:11.038336 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:11.038224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxrx9\" (UniqueName: \"kubernetes.io/projected/7a2ad887-2bf1-4008-a61a-f7092ba4885d-kube-api-access-qxrx9\") pod \"payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:11.038799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:11.038779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2ad887-2bf1-4008-a61a-f7092ba4885d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:11.047551 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:11.047525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxrx9\" (UniqueName: \"kubernetes.io/projected/7a2ad887-2bf1-4008-a61a-f7092ba4885d-kube-api-access-qxrx9\") pod \"payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:11.212196 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:11.212156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:11.486885 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:11.486851    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"]
Sep 11 23:36:11.489378 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:36:11.489352    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2ad887_2bf1_4008_a61a_f7092ba4885d.slice/crio-c559ca8be52c54060b69473fd94e224031d175064421e334b988acb4175ee1fb WatchSource:0}: Error finding container c559ca8be52c54060b69473fd94e224031d175064421e334b988acb4175ee1fb: Status 404 returned error can't find the container with id c559ca8be52c54060b69473fd94e224031d175064421e334b988acb4175ee1fb
Sep 11 23:36:12.424101 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:12.424064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4" event={"ID":"7a2ad887-2bf1-4008-a61a-f7092ba4885d","Type":"ContainerStarted","Data":"4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe"}
Sep 11 23:36:12.424286 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:12.424121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4" event={"ID":"7a2ad887-2bf1-4008-a61a-f7092ba4885d","Type":"ContainerStarted","Data":"c559ca8be52c54060b69473fd94e224031d175064421e334b988acb4175ee1fb"}
Sep 11 23:36:12.424286 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:12.424173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:12.451567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:12.451518    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4" podStartSLOduration=2.451502043 podStartE2EDuration="2.451502043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:36:12.450420834 +0000 UTC m=+2087.662392021" watchObservedRunningTime="2026-09-11 23:36:12.451502043 +0000 UTC m=+2087.663473218"
Sep 11 23:36:23.430459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:23.430424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:36:25.750462 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:25.750425    2580 scope.go:117] "RemoveContainer" containerID="ac19708a7873449a9cf2e7a7fcdb0c8b2b3dd5a5df758c90a2b2faebec5a0934"
Sep 11 23:36:32.770859 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:32.770826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:33.266150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:33.266112    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:33.877066 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:33.877038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:33.977600 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:33.977571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:34.216358 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:34.216276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:34.326908 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:34.326873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:34.663010 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:34.662974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:35.489757 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:35.489718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:36.412337 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:36.412300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:36.900661 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:36.900626    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:37.565535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:37.565499    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:37.658460 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:37.658423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:37.759988 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:37.759952    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:38.668445 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:38.668399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:39.391751 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:39.391710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:40.138942 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:40.138898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:40.201399 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:40.201365    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:40.358707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:40.358674    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:40.815114 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:40.815081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:40.912176 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:40.912144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:41.000284 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.000255    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"]
Sep 11 23:36:41.003427 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.003409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.006354 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.006335    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-778566\""
Sep 11 23:36:41.006599 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.006579    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-778566-dockercfg-gjm2d\""
Sep 11 23:36:41.015878 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.015856    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"]
Sep 11 23:36:41.091990 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.091889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4d9w\" (UniqueName: \"kubernetes.io/projected/e9740a1a-23d9-46df-9bcd-28c76ab225db-kube-api-access-m4d9w\") pod \"payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.092140 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.091995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9740a1a-23d9-46df-9bcd-28c76ab225db-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.193328 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.193295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4d9w\" (UniqueName: \"kubernetes.io/projected/e9740a1a-23d9-46df-9bcd-28c76ab225db-kube-api-access-m4d9w\") pod \"payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.193678 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.193345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9740a1a-23d9-46df-9bcd-28c76ab225db-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.194058 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.194040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9740a1a-23d9-46df-9bcd-28c76ab225db-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.204310 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.204286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4d9w\" (UniqueName: \"kubernetes.io/projected/e9740a1a-23d9-46df-9bcd-28c76ab225db-kube-api-access-m4d9w\") pod \"payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.264221 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.264176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:41.314045 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.314009    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:41.593264 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:41.593235    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"]
Sep 11 23:36:41.594710 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:36:41.594683    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9740a1a_23d9_46df_9bcd_28c76ab225db.slice/crio-dbae2f0581be7292a981f443a7d9612f2fea4775ee45350493a46db3b81808a6 WatchSource:0}: Error finding container dbae2f0581be7292a981f443a7d9612f2fea4775ee45350493a46db3b81808a6: Status 404 returned error can't find the container with id dbae2f0581be7292a981f443a7d9612f2fea4775ee45350493a46db3b81808a6
Sep 11 23:36:42.245676 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:42.245635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:36:42.558336 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:42.558238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx" event={"ID":"e9740a1a-23d9-46df-9bcd-28c76ab225db","Type":"ContainerStarted","Data":"ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a"}
Sep 11 23:36:42.558336 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:42.558289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx" event={"ID":"e9740a1a-23d9-46df-9bcd-28c76ab225db","Type":"ContainerStarted","Data":"dbae2f0581be7292a981f443a7d9612f2fea4775ee45350493a46db3b81808a6"}
Sep 11 23:36:42.558572 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:42.558341    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:36:42.575969 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:42.575903    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx" podStartSLOduration=2.575886394 podStartE2EDuration="2.575886394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:36:42.575058803 +0000 UTC m=+2117.787029978" watchObservedRunningTime="2026-09-11 23:36:42.575886394 +0000 UTC m=+2117.787857568"
Sep 11 23:36:53.565130 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:36:53.565099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:37:02.539788 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:02.539747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:04.053441 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:04.053405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:04.183620 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:04.183585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:04.924276 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:04.924244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:05.681370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:05.681337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:07.474587 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:07.474555    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6cdvp_5174e8da-57b9-402d-911d-c03550f89619/payload-processing/0.log"
Sep 11 23:37:30.825785 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:30.825703    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:55.668016 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:55.667933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4_7a2ad887-2bf1-4008-a61a-f7092ba4885d/payload-processing/0.log"
Sep 11 23:37:55.795459 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:55.795434    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6cdvp_5174e8da-57b9-402d-911d-c03550f89619/payload-processing/0.log"
Sep 11 23:37:56.181492 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:56.181457    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:56.476897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:56.476815    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:57.152058 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:57.152009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:58.391473 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:58.391439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:59.065142 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:59.065110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:59.186795 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:59.186761    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:37:59.463319 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:37:59.463224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:00.340703 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:00.340662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:01.133436 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:01.133396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:01.904077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:01.904046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:02.780251 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:02.780217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:02.888612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:02.888574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:03.111996 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.111964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:03.377751 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.377664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:03.476024 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.475985    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:03.487630 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.487594    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"]
Sep 11 23:38:03.491369 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.491349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.494147 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.494127    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-e231ef\""
Sep 11 23:38:03.494277 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.494240    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-e231ef-dockercfg-fxb7p\""
Sep 11 23:38:03.502219 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.502194    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"]
Sep 11 23:38:03.512225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.512192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdsvd\" (UniqueName: \"kubernetes.io/projected/89aca76d-7e45-4057-8f40-c94ccf1cf09c-kube-api-access-gdsvd\") pod \"payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.512367 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.512253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89aca76d-7e45-4057-8f40-c94ccf1cf09c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.521948 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.521902    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"]
Sep 11 23:38:03.525970 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.525944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:03.536205 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.536180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"]
Sep 11 23:38:03.613110 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.613073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5335471-9c0d-4548-b808-75a8a73dd7c2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:03.613278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.613148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdsvd\" (UniqueName: \"kubernetes.io/projected/89aca76d-7e45-4057-8f40-c94ccf1cf09c-kube-api-access-gdsvd\") pod \"payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.613278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.613179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfhg6\" (UniqueName: \"kubernetes.io/projected/e5335471-9c0d-4548-b808-75a8a73dd7c2-kube-api-access-lfhg6\") pod \"payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:03.613278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.613229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89aca76d-7e45-4057-8f40-c94ccf1cf09c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.613896 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.613877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89aca76d-7e45-4057-8f40-c94ccf1cf09c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.621683 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.621655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdsvd\" (UniqueName: \"kubernetes.io/projected/89aca76d-7e45-4057-8f40-c94ccf1cf09c-kube-api-access-gdsvd\") pod \"payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.714401 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.714299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5335471-9c0d-4548-b808-75a8a73dd7c2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:03.714401 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.714379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfhg6\" (UniqueName: \"kubernetes.io/projected/e5335471-9c0d-4548-b808-75a8a73dd7c2-kube-api-access-lfhg6\") pod \"payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:03.715053 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.715034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5335471-9c0d-4548-b808-75a8a73dd7c2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:03.722868 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.722841    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfhg6\" (UniqueName: \"kubernetes.io/projected/e5335471-9c0d-4548-b808-75a8a73dd7c2-kube-api-access-lfhg6\") pod \"payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:03.802502 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.802465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:03.835999 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:03.835965    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:04.077562 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.077538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"]
Sep 11 23:38:04.078863 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:38:04.078826    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89aca76d_7e45_4057_8f40_c94ccf1cf09c.slice/crio-8d50f02b46e0186cd7149d8c3db3e4bd49bbefa3fd255eee655c07b6c9409cdf WatchSource:0}: Error finding container 8d50f02b46e0186cd7149d8c3db3e4bd49bbefa3fd255eee655c07b6c9409cdf: Status 404 returned error can't find the container with id 8d50f02b46e0186cd7149d8c3db3e4bd49bbefa3fd255eee655c07b6c9409cdf
Sep 11 23:38:04.122042 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.122016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"]
Sep 11 23:38:04.123884 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:38:04.123859    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5335471_9c0d_4548_b808_75a8a73dd7c2.slice/crio-395594310e19b0ac7f42624864d31df82cfaa22f8472264a54af5e3198c110b7 WatchSource:0}: Error finding container 395594310e19b0ac7f42624864d31df82cfaa22f8472264a54af5e3198c110b7: Status 404 returned error can't find the container with id 395594310e19b0ac7f42624864d31df82cfaa22f8472264a54af5e3198c110b7
Sep 11 23:38:04.938675 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.938640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7" event={"ID":"e5335471-9c0d-4548-b808-75a8a73dd7c2","Type":"ContainerStarted","Data":"355ed613b67c0f0484f7430ad24d5d66e43b5caf60ec9f7b17c1c8691fe7aec9"}
Sep 11 23:38:04.939165 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.938702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7" event={"ID":"e5335471-9c0d-4548-b808-75a8a73dd7c2","Type":"ContainerStarted","Data":"395594310e19b0ac7f42624864d31df82cfaa22f8472264a54af5e3198c110b7"}
Sep 11 23:38:04.939165 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.938845    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:04.940244 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.940219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv" event={"ID":"89aca76d-7e45-4057-8f40-c94ccf1cf09c","Type":"ContainerStarted","Data":"d999bd69a1c4758b05beb8fd0e3432c6adba3702a81158001aba24b100c59067"}
Sep 11 23:38:04.940352 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.940255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv" event={"ID":"89aca76d-7e45-4057-8f40-c94ccf1cf09c","Type":"ContainerStarted","Data":"8d50f02b46e0186cd7149d8c3db3e4bd49bbefa3fd255eee655c07b6c9409cdf"}
Sep 11 23:38:04.940405 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.940358    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:04.957010 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.956942    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7" podStartSLOduration=1.956899033 podStartE2EDuration="1.956899033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:38:04.955972808 +0000 UTC m=+2200.167943982" watchObservedRunningTime="2026-09-11 23:38:04.956899033 +0000 UTC m=+2200.168870208"
Sep 11 23:38:04.981690 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:04.981627    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv" podStartSLOduration=1.981607523 podStartE2EDuration="1.981607523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:38:04.981160069 +0000 UTC m=+2200.193131243" watchObservedRunningTime="2026-09-11 23:38:04.981607523 +0000 UTC m=+2200.193578701"
Sep 11 23:38:15.947069 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:15.947037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:15.947463 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:15.947110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:24.876903 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:24.876862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:25.774836 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:25.774804    2580 scope.go:117] "RemoveContainer" containerID="615c8de6c16639ce36c145c433980799660bc07cd1b199eb2e24d921a5c9130b"
Sep 11 23:38:25.781804 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:25.781776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:26.457832 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:26.457791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:27.334327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:27.334287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:28.047386 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:28.047353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:28.731052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:28.731013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:29.141515 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:29.141484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:30.693694 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:30.693661    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:31.462638 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:31.462602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:32.278248 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:32.278210    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:32.995708 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:32.995670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:33.846306 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:33.846274    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:34.568949 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:34.568900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:40.936714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:40.936680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:40.969050 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:40.969017    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"]
Sep 11 23:38:40.969305 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:40.969279    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7" podUID="e5335471-9c0d-4548-b808-75a8a73dd7c2" containerName="payload-processing" containerID="cri-o://355ed613b67c0f0484f7430ad24d5d66e43b5caf60ec9f7b17c1c8691fe7aec9" gracePeriod=30
Sep 11 23:38:40.979726 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:40.979699    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"]
Sep 11 23:38:40.979964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:40.979933    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv" podUID="89aca76d-7e45-4057-8f40-c94ccf1cf09c" containerName="payload-pre-processing" containerID="cri-o://d999bd69a1c4758b05beb8fd0e3432c6adba3702a81158001aba24b100c59067" gracePeriod=30
Sep 11 23:38:41.018639 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.018609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:41.079614 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.079579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:41.107109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.107038    2580 generic.go:356] "Generic (PLEG): container finished" podID="e5335471-9c0d-4548-b808-75a8a73dd7c2" containerID="355ed613b67c0f0484f7430ad24d5d66e43b5caf60ec9f7b17c1c8691fe7aec9" exitCode=0
Sep 11 23:38:41.107298 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.107116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7" event={"ID":"e5335471-9c0d-4548-b808-75a8a73dd7c2","Type":"ContainerDied","Data":"355ed613b67c0f0484f7430ad24d5d66e43b5caf60ec9f7b17c1c8691fe7aec9"}
Sep 11 23:38:41.108755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.108685    2580 generic.go:356] "Generic (PLEG): container finished" podID="89aca76d-7e45-4057-8f40-c94ccf1cf09c" containerID="d999bd69a1c4758b05beb8fd0e3432c6adba3702a81158001aba24b100c59067" exitCode=0
Sep 11 23:38:41.108864 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.108756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv" event={"ID":"89aca76d-7e45-4057-8f40-c94ccf1cf09c","Type":"ContainerDied","Data":"d999bd69a1c4758b05beb8fd0e3432c6adba3702a81158001aba24b100c59067"}
Sep 11 23:38:41.167886 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.167835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:41.217370 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.217345    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:41.240587 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.240551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:41.256758 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.256737    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:41.339054 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.339017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89aca76d-7e45-4057-8f40-c94ccf1cf09c-plugins-config-volume\") pod \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") "
Sep 11 23:38:41.339225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.339084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdsvd\" (UniqueName: \"kubernetes.io/projected/89aca76d-7e45-4057-8f40-c94ccf1cf09c-kube-api-access-gdsvd\") pod \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\" (UID: \"89aca76d-7e45-4057-8f40-c94ccf1cf09c\") "
Sep 11 23:38:41.339225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.339114    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5335471-9c0d-4548-b808-75a8a73dd7c2-plugins-config-volume\") pod \"e5335471-9c0d-4548-b808-75a8a73dd7c2\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") "
Sep 11 23:38:41.339225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.339152    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfhg6\" (UniqueName: \"kubernetes.io/projected/e5335471-9c0d-4548-b808-75a8a73dd7c2-kube-api-access-lfhg6\") pod \"e5335471-9c0d-4548-b808-75a8a73dd7c2\" (UID: \"e5335471-9c0d-4548-b808-75a8a73dd7c2\") "
Sep 11 23:38:41.339507 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.339471    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89aca76d-7e45-4057-8f40-c94ccf1cf09c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89aca76d-7e45-4057-8f40-c94ccf1cf09c" (UID: "89aca76d-7e45-4057-8f40-c94ccf1cf09c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:38:41.339507 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.339501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5335471-9c0d-4548-b808-75a8a73dd7c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5335471-9c0d-4548-b808-75a8a73dd7c2" (UID: "e5335471-9c0d-4548-b808-75a8a73dd7c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:38:41.341228 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.341209    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5335471-9c0d-4548-b808-75a8a73dd7c2-kube-api-access-lfhg6" (OuterVolumeSpecName: "kube-api-access-lfhg6") pod "e5335471-9c0d-4548-b808-75a8a73dd7c2" (UID: "e5335471-9c0d-4548-b808-75a8a73dd7c2"). InnerVolumeSpecName "kube-api-access-lfhg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:38:41.341303 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.341292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89aca76d-7e45-4057-8f40-c94ccf1cf09c-kube-api-access-gdsvd" (OuterVolumeSpecName: "kube-api-access-gdsvd") pod "89aca76d-7e45-4057-8f40-c94ccf1cf09c" (UID: "89aca76d-7e45-4057-8f40-c94ccf1cf09c"). InnerVolumeSpecName "kube-api-access-gdsvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:38:41.440767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.440733    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89aca76d-7e45-4057-8f40-c94ccf1cf09c-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.440767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.440761    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdsvd\" (UniqueName: \"kubernetes.io/projected/89aca76d-7e45-4057-8f40-c94ccf1cf09c-kube-api-access-gdsvd\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.440767 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.440771    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5335471-9c0d-4548-b808-75a8a73dd7c2-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.441006 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.440779    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfhg6\" (UniqueName: \"kubernetes.io/projected/e5335471-9c0d-4548-b808-75a8a73dd7c2-kube-api-access-lfhg6\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.530979 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.530871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:41.631981 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:41.631939    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:42.114775 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.114743    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"
Sep 11 23:38:42.114775 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.114765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv" event={"ID":"89aca76d-7e45-4057-8f40-c94ccf1cf09c","Type":"ContainerDied","Data":"8d50f02b46e0186cd7149d8c3db3e4bd49bbefa3fd255eee655c07b6c9409cdf"}
Sep 11 23:38:42.115322 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.114828    2580 scope.go:117] "RemoveContainer" containerID="d999bd69a1c4758b05beb8fd0e3432c6adba3702a81158001aba24b100c59067"
Sep 11 23:38:42.116495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.116471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7" event={"ID":"e5335471-9c0d-4548-b808-75a8a73dd7c2","Type":"ContainerDied","Data":"395594310e19b0ac7f42624864d31df82cfaa22f8472264a54af5e3198c110b7"}
Sep 11 23:38:42.116495 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.116491    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"
Sep 11 23:38:42.124225 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.124205    2580 scope.go:117] "RemoveContainer" containerID="355ed613b67c0f0484f7430ad24d5d66e43b5caf60ec9f7b17c1c8691fe7aec9"
Sep 11 23:38:42.138577 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.138550    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"]
Sep 11 23:38:42.142985 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.142961    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e231ef-6cb86f4574-t8hjv"]
Sep 11 23:38:42.160625 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.160584    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"]
Sep 11 23:38:42.160816 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.160653    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e231ef-845c747955-qjnr7"]
Sep 11 23:38:42.455532 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:42.455495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:43.289527 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:43.289490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:43.413891 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:43.413860    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89aca76d-7e45-4057-8f40-c94ccf1cf09c" path="/var/lib/kubelet/pods/89aca76d-7e45-4057-8f40-c94ccf1cf09c/volumes"
Sep 11 23:38:43.414278 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:43.414263    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5335471-9c0d-4548-b808-75a8a73dd7c2" path="/var/lib/kubelet/pods/e5335471-9c0d-4548-b808-75a8a73dd7c2/volumes"
Sep 11 23:38:43.952220 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:43.952185    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:44.678975 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:44.678898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:47.383893 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:47.383856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:47.497025 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:47.496987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:47.678216 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:47.678137    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:51.093663 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.093626    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"]
Sep 11 23:38:51.094062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.093981    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5335471-9c0d-4548-b808-75a8a73dd7c2" containerName="payload-processing"
Sep 11 23:38:51.094062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.093995    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5335471-9c0d-4548-b808-75a8a73dd7c2" containerName="payload-processing"
Sep 11 23:38:51.094062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.094006    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89aca76d-7e45-4057-8f40-c94ccf1cf09c" containerName="payload-pre-processing"
Sep 11 23:38:51.094062 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.094011    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="89aca76d-7e45-4057-8f40-c94ccf1cf09c" containerName="payload-pre-processing"
Sep 11 23:38:51.094200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.094081    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5335471-9c0d-4548-b808-75a8a73dd7c2" containerName="payload-processing"
Sep 11 23:38:51.094200 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.094092    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="89aca76d-7e45-4057-8f40-c94ccf1cf09c" containerName="payload-pre-processing"
Sep 11 23:38:51.097254 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.097237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"
Sep 11 23:38:51.100306 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.100291    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 23:38:51.100376 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.100294    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vrx7m\""
Sep 11 23:38:51.105195 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.105172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"]
Sep 11 23:38:51.228806 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.228762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb\" (UID: \"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"
Sep 11 23:38:51.330312 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.330263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb\" (UID: \"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"
Sep 11 23:38:51.330903 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.330881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb\" (UID: \"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"
Sep 11 23:38:51.408408 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.408320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"
Sep 11 23:38:51.457816 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.457781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:51.680773 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:51.680749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"]
Sep 11 23:38:51.682887 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:38:51.682855    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5373c17_b2a5_4e6b_9e9b_13cf2e32ba5e.slice/crio-5fac011a592643ec2fa2deb716184a1b149cea3da20a5e414d00de5be2279ae9 WatchSource:0}: Error finding container 5fac011a592643ec2fa2deb716184a1b149cea3da20a5e414d00de5be2279ae9: Status 404 returned error can't find the container with id 5fac011a592643ec2fa2deb716184a1b149cea3da20a5e414d00de5be2279ae9
Sep 11 23:38:52.168606 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:52.168569    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb" event={"ID":"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e","Type":"ContainerStarted","Data":"33c3beea776b83dd1463b91d3c1a738d9ae90eadffd70fcbee3f647bdeeadd0d"}
Sep 11 23:38:52.169011 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:52.168620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb" event={"ID":"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e","Type":"ContainerStarted","Data":"5fac011a592643ec2fa2deb716184a1b149cea3da20a5e414d00de5be2279ae9"}
Sep 11 23:38:52.191308 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:52.191249    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb" podStartSLOduration=1.191230131 podStartE2EDuration="1.191230131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:38:52.190243255 +0000 UTC m=+2247.402214429" watchObservedRunningTime="2026-09-11 23:38:52.191230131 +0000 UTC m=+2247.403201305"
Sep 11 23:38:52.475564 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:52.475469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:53.174781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:53.174749    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e" containerID="33c3beea776b83dd1463b91d3c1a738d9ae90eadffd70fcbee3f647bdeeadd0d" exitCode=0
Sep 11 23:38:53.175183 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:53.174841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb" event={"ID":"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e","Type":"ContainerDied","Data":"33c3beea776b83dd1463b91d3c1a738d9ae90eadffd70fcbee3f647bdeeadd0d"}
Sep 11 23:38:53.287891 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:53.287833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:53.991669 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:53.991632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:54.308596 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:54.308564    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"
Sep 11 23:38:54.355222 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:54.355186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e-maas-api-service-ca\") pod \"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e\" (UID: \"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e\") "
Sep 11 23:38:54.355580 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:54.355558    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e" (UID: "d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:38:54.456209 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:54.456173    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e-maas-api-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:38:54.849169 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:54.849130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:38:55.186677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:55.186596    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"
Sep 11 23:38:55.186843 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:55.186594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb" event={"ID":"d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e","Type":"ContainerDied","Data":"5fac011a592643ec2fa2deb716184a1b149cea3da20a5e414d00de5be2279ae9"}
Sep 11 23:38:55.186843 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:55.186735    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fac011a592643ec2fa2deb716184a1b149cea3da20a5e414d00de5be2279ae9"
Sep 11 23:38:55.605516 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:38:55.605479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:06.689629 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.689585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:06.721652 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.721615    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"]
Sep 11 23:39:06.721895 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.721870    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4" podUID="7a2ad887-2bf1-4008-a61a-f7092ba4885d" containerName="payload-processing" containerID="cri-o://4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe" gracePeriod=30
Sep 11 23:39:06.791615 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.791579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:06.893967 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.893904    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:06.901803 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.901773    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"]
Sep 11 23:39:06.904448 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.904420    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1fb90b-shtrb"]
Sep 11 23:39:06.959467 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.959429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:06.976294 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:06.976269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:39:07.064954 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.064893    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxrx9\" (UniqueName: \"kubernetes.io/projected/7a2ad887-2bf1-4008-a61a-f7092ba4885d-kube-api-access-qxrx9\") pod \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") "
Sep 11 23:39:07.065148 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.064993    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2ad887-2bf1-4008-a61a-f7092ba4885d-plugins-config-volume\") pod \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\" (UID: \"7a2ad887-2bf1-4008-a61a-f7092ba4885d\") "
Sep 11 23:39:07.065475 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.065437    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a2ad887-2bf1-4008-a61a-f7092ba4885d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a2ad887-2bf1-4008-a61a-f7092ba4885d" (UID: "7a2ad887-2bf1-4008-a61a-f7092ba4885d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:39:07.067192 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.067159    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2ad887-2bf1-4008-a61a-f7092ba4885d-kube-api-access-qxrx9" (OuterVolumeSpecName: "kube-api-access-qxrx9") pod "7a2ad887-2bf1-4008-a61a-f7092ba4885d" (UID: "7a2ad887-2bf1-4008-a61a-f7092ba4885d"). InnerVolumeSpecName "kube-api-access-qxrx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:39:07.166662 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.166605    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxrx9\" (UniqueName: \"kubernetes.io/projected/7a2ad887-2bf1-4008-a61a-f7092ba4885d-kube-api-access-qxrx9\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.166662 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.166654    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2ad887-2bf1-4008-a61a-f7092ba4885d-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.243655 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.243558    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a2ad887-2bf1-4008-a61a-f7092ba4885d" containerID="4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe" exitCode=0
Sep 11 23:39:07.243655 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.243606    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"
Sep 11 23:39:07.243891 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.243643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4" event={"ID":"7a2ad887-2bf1-4008-a61a-f7092ba4885d","Type":"ContainerDied","Data":"4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe"}
Sep 11 23:39:07.243891 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.243712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4" event={"ID":"7a2ad887-2bf1-4008-a61a-f7092ba4885d","Type":"ContainerDied","Data":"c559ca8be52c54060b69473fd94e224031d175064421e334b988acb4175ee1fb"}
Sep 11 23:39:07.243891 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.243756    2580 scope.go:117] "RemoveContainer" containerID="4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe"
Sep 11 23:39:07.253477 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.253457    2580 scope.go:117] "RemoveContainer" containerID="4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe"
Sep 11 23:39:07.253756 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:39:07.253737    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe\": container with ID starting with 4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe not found: ID does not exist" containerID="4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe"
Sep 11 23:39:07.253798 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.253766    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe"} err="failed to get container status \"4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe\": rpc error: code = NotFound desc = could not find container \"4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe\": container with ID starting with 4b8043abf5e995709eec09f410b89d22f591daa62c405bd7917f1e33d9932cbe not found: ID does not exist"
Sep 11 23:39:07.265939 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.265893    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"]
Sep 11 23:39:07.269454 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.269430    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1fb90b-cfbb678c7-mxtv4"]
Sep 11 23:39:07.289432 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.289403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:07.348679 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.348636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:07.413475 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.413441    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2ad887-2bf1-4008-a61a-f7092ba4885d" path="/var/lib/kubelet/pods/7a2ad887-2bf1-4008-a61a-f7092ba4885d/volumes"
Sep 11 23:39:07.413825 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:07.413809    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e" path="/var/lib/kubelet/pods/d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e/volumes"
Sep 11 23:39:08.065571 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:08.065524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:08.744016 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:08.743964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:09.556695 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:09.556644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:12.360544 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:12.360511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:42.215154 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:42.215118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:42.276265 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:42.276233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:49.024748 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:49.024709    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:49.259348 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:49.259318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:39:54.706513 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:39:54.706474    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:08.861172 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:08.861128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:08.964020 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:08.963984    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:09.149600 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:09.149488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:09.663029 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:09.662992    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:10.469984 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:10.469944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:11.358428 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:11.358393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:12.051156 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:12.051125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:12.742000 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:12.741964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:19.994732 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:19.994622    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:20.013978 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.013946    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"]
Sep 11 23:40:20.014172 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.014146    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx" podUID="e9740a1a-23d9-46df-9bcd-28c76ab225db" containerName="payload-pre-processing" containerID="cri-o://ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a" gracePeriod=30
Sep 11 23:40:20.093205 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.093170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:20.177215 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.177176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:20.263290 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.263263    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:40:20.284472 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.284441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:20.405221 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.405175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9740a1a-23d9-46df-9bcd-28c76ab225db-plugins-config-volume\") pod \"e9740a1a-23d9-46df-9bcd-28c76ab225db\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") "
Sep 11 23:40:20.405397 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.405300    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4d9w\" (UniqueName: \"kubernetes.io/projected/e9740a1a-23d9-46df-9bcd-28c76ab225db-kube-api-access-m4d9w\") pod \"e9740a1a-23d9-46df-9bcd-28c76ab225db\" (UID: \"e9740a1a-23d9-46df-9bcd-28c76ab225db\") "
Sep 11 23:40:20.405559 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.405536    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9740a1a-23d9-46df-9bcd-28c76ab225db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9740a1a-23d9-46df-9bcd-28c76ab225db" (UID: "e9740a1a-23d9-46df-9bcd-28c76ab225db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:20.407365 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.407336    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9740a1a-23d9-46df-9bcd-28c76ab225db-kube-api-access-m4d9w" (OuterVolumeSpecName: "kube-api-access-m4d9w") pod "e9740a1a-23d9-46df-9bcd-28c76ab225db" (UID: "e9740a1a-23d9-46df-9bcd-28c76ab225db"). InnerVolumeSpecName "kube-api-access-m4d9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:20.506827 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.506751    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9740a1a-23d9-46df-9bcd-28c76ab225db-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.506827 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.506777    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4d9w\" (UniqueName: \"kubernetes.io/projected/e9740a1a-23d9-46df-9bcd-28c76ab225db-kube-api-access-m4d9w\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.535007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.534971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:20.584367 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.584330    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9740a1a-23d9-46df-9bcd-28c76ab225db" containerID="ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a" exitCode=0
Sep 11 23:40:20.584535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.584377    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"
Sep 11 23:40:20.584535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.584421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx" event={"ID":"e9740a1a-23d9-46df-9bcd-28c76ab225db","Type":"ContainerDied","Data":"ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a"}
Sep 11 23:40:20.584535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.584467    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx" event={"ID":"e9740a1a-23d9-46df-9bcd-28c76ab225db","Type":"ContainerDied","Data":"dbae2f0581be7292a981f443a7d9612f2fea4775ee45350493a46db3b81808a6"}
Sep 11 23:40:20.584535 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.584490    2580 scope.go:117] "RemoveContainer" containerID="ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a"
Sep 11 23:40:20.593340 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.593316    2580 scope.go:117] "RemoveContainer" containerID="ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a"
Sep 11 23:40:20.593648 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:40:20.593628    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a\": container with ID starting with ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a not found: ID does not exist" containerID="ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a"
Sep 11 23:40:20.593703 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.593662    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a"} err="failed to get container status \"ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a\": rpc error: code = NotFound desc = could not find container \"ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a\": container with ID starting with ebe132afe103e1f723a300505fb7cc4ae1016da1c0aea7f5dbc405628986b05a not found: ID does not exist"
Sep 11 23:40:20.606058 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.606029    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"]
Sep 11 23:40:20.611433 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.611391    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-778566-58bc558dbb-7mfnx"]
Sep 11 23:40:20.696953 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:20.696901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:21.414138 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:21.414098    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9740a1a-23d9-46df-9bcd-28c76ab225db" path="/var/lib/kubelet/pods/e9740a1a-23d9-46df-9bcd-28c76ab225db/volumes"
Sep 11 23:40:21.442545 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:21.442510    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:22.255319 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:22.255278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:26.381569 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:26.381530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:26.481437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:26.481405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:26.642466 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:26.642379    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:27.074436 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:27.074406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:27.960102 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:27.960069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:28.760856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:28.760824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:29.468232 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:29.468196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:37.616376 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.616338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:37.621109 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.621080    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"]
Sep 11 23:40:37.624810 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.624783    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-1c4628-29819490-kg8qh"]
Sep 11 23:40:37.645846 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.645815    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"]
Sep 11 23:40:37.646051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.646029    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px" podUID="c1519560-2009-43ad-ba17-59b74e4cb7cd" containerName="payload-processing" containerID="cri-o://75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592" gracePeriod=30
Sep 11 23:40:37.658666 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.658634    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"]
Sep 11 23:40:37.659051 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.658907    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w" podUID="d1da8b65-21d5-42c8-826b-3a3f31169755" containerName="payload-pre-processing" containerID="cri-o://ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85" gracePeriod=30
Sep 11 23:40:37.690882 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.690845    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:37.746410 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.746378    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:37.828417 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.828382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:37.892496 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.892402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:37.929534 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.929506    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:40:37.931510 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.931491    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:40:37.957353 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.957324    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1da8b65-21d5-42c8-826b-3a3f31169755-plugins-config-volume\") pod \"d1da8b65-21d5-42c8-826b-3a3f31169755\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") "
Sep 11 23:40:37.957540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.957361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1519560-2009-43ad-ba17-59b74e4cb7cd-plugins-config-volume\") pod \"c1519560-2009-43ad-ba17-59b74e4cb7cd\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") "
Sep 11 23:40:37.957540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.957416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2s8z\" (UniqueName: \"kubernetes.io/projected/d1da8b65-21d5-42c8-826b-3a3f31169755-kube-api-access-x2s8z\") pod \"d1da8b65-21d5-42c8-826b-3a3f31169755\" (UID: \"d1da8b65-21d5-42c8-826b-3a3f31169755\") "
Sep 11 23:40:37.957540 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.957474    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fkf7\" (UniqueName: \"kubernetes.io/projected/c1519560-2009-43ad-ba17-59b74e4cb7cd-kube-api-access-4fkf7\") pod \"c1519560-2009-43ad-ba17-59b74e4cb7cd\" (UID: \"c1519560-2009-43ad-ba17-59b74e4cb7cd\") "
Sep 11 23:40:37.957774 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.957751    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1da8b65-21d5-42c8-826b-3a3f31169755-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1da8b65-21d5-42c8-826b-3a3f31169755" (UID: "d1da8b65-21d5-42c8-826b-3a3f31169755"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:37.957822 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.957784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1519560-2009-43ad-ba17-59b74e4cb7cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1519560-2009-43ad-ba17-59b74e4cb7cd" (UID: "c1519560-2009-43ad-ba17-59b74e4cb7cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:37.959733 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.959707    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1da8b65-21d5-42c8-826b-3a3f31169755-kube-api-access-x2s8z" (OuterVolumeSpecName: "kube-api-access-x2s8z") pod "d1da8b65-21d5-42c8-826b-3a3f31169755" (UID: "d1da8b65-21d5-42c8-826b-3a3f31169755"). InnerVolumeSpecName "kube-api-access-x2s8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:37.959821 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:37.959744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1519560-2009-43ad-ba17-59b74e4cb7cd-kube-api-access-4fkf7" (OuterVolumeSpecName: "kube-api-access-4fkf7") pod "c1519560-2009-43ad-ba17-59b74e4cb7cd" (UID: "c1519560-2009-43ad-ba17-59b74e4cb7cd"). InnerVolumeSpecName "kube-api-access-4fkf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:38.058906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.058874    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fkf7\" (UniqueName: \"kubernetes.io/projected/c1519560-2009-43ad-ba17-59b74e4cb7cd-kube-api-access-4fkf7\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.058906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.058902    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1da8b65-21d5-42c8-826b-3a3f31169755-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.058906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.058933    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1519560-2009-43ad-ba17-59b74e4cb7cd-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.059146 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.058942    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2s8z\" (UniqueName: \"kubernetes.io/projected/d1da8b65-21d5-42c8-826b-3a3f31169755-kube-api-access-x2s8z\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.190569 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.190484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:38.291691 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.291656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:38.674058 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.674027    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1da8b65-21d5-42c8-826b-3a3f31169755" containerID="ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85" exitCode=0
Sep 11 23:40:38.674508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.674087    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"
Sep 11 23:40:38.674508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.674109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w" event={"ID":"d1da8b65-21d5-42c8-826b-3a3f31169755","Type":"ContainerDied","Data":"ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85"}
Sep 11 23:40:38.674508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.674163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w" event={"ID":"d1da8b65-21d5-42c8-826b-3a3f31169755","Type":"ContainerDied","Data":"e74598e40bfc95005d2c09b0c4c00cf5a6c446a90e27851dde894bd2b5d8a579"}
Sep 11 23:40:38.674508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.674177    2580 scope.go:117] "RemoveContainer" containerID="ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85"
Sep 11 23:40:38.675586 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.675484    2580 generic.go:356] "Generic (PLEG): container finished" podID="c1519560-2009-43ad-ba17-59b74e4cb7cd" containerID="75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592" exitCode=0
Sep 11 23:40:38.675586 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.675525    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"
Sep 11 23:40:38.675714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.675522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px" event={"ID":"c1519560-2009-43ad-ba17-59b74e4cb7cd","Type":"ContainerDied","Data":"75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592"}
Sep 11 23:40:38.675714 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.675661    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px" event={"ID":"c1519560-2009-43ad-ba17-59b74e4cb7cd","Type":"ContainerDied","Data":"34945b57674a61d5533b5f03d0b1f893ce0e59ae7e1f0abada6502ae730cf5e2"}
Sep 11 23:40:38.684261 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.684243    2580 scope.go:117] "RemoveContainer" containerID="ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85"
Sep 11 23:40:38.684563 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:40:38.684542    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85\": container with ID starting with ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85 not found: ID does not exist" containerID="ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85"
Sep 11 23:40:38.684662 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.684568    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85"} err="failed to get container status \"ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85\": rpc error: code = NotFound desc = could not find container \"ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85\": container with ID starting with ae587dd62284a14d576f642d849535c8f257b079e462338d29a2258ff0dfef85 not found: ID does not exist"
Sep 11 23:40:38.684662 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.684584    2580 scope.go:117] "RemoveContainer" containerID="75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592"
Sep 11 23:40:38.692272 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.692256    2580 scope.go:117] "RemoveContainer" containerID="75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592"
Sep 11 23:40:38.692563 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:40:38.692545    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592\": container with ID starting with 75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592 not found: ID does not exist" containerID="75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592"
Sep 11 23:40:38.692612 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.692571    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592"} err="failed to get container status \"75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592\": rpc error: code = NotFound desc = could not find container \"75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592\": container with ID starting with 75d28fade0ba323d2b2bfa8b618a3631a3cf060505c7751d060ca11f38705592 not found: ID does not exist"
Sep 11 23:40:38.699706 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.699683    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"]
Sep 11 23:40:38.703237 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.703218    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1c4628-6d4478c59d-tfs2w"]
Sep 11 23:40:38.714423 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.714402    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"]
Sep 11 23:40:38.718552 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:38.718530    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-1c4628-75dbdcf85f-n86px"]
Sep 11 23:40:39.074096 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:39.074065    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:39.413534 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:39.413448    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3efb1017-78f3-4e20-9a97-9a1ca2680f2a" path="/var/lib/kubelet/pods/3efb1017-78f3-4e20-9a97-9a1ca2680f2a/volumes"
Sep 11 23:40:39.413962 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:39.413944    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1519560-2009-43ad-ba17-59b74e4cb7cd" path="/var/lib/kubelet/pods/c1519560-2009-43ad-ba17-59b74e4cb7cd/volumes"
Sep 11 23:40:39.414288 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:39.414276    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1da8b65-21d5-42c8-826b-3a3f31169755" path="/var/lib/kubelet/pods/d1da8b65-21d5-42c8-826b-3a3f31169755/volumes"
Sep 11 23:40:44.153070 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.153035    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:44.297437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.297400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:44.435441 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.435346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:44.766123 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766032    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"]
Sep 11 23:40:44.766458 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766440    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2ad887-2bf1-4008-a61a-f7092ba4885d" containerName="payload-processing"
Sep 11 23:40:44.766537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766460    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2ad887-2bf1-4008-a61a-f7092ba4885d" containerName="payload-processing"
Sep 11 23:40:44.766537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766479    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1519560-2009-43ad-ba17-59b74e4cb7cd" containerName="payload-processing"
Sep 11 23:40:44.766537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766487    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1519560-2009-43ad-ba17-59b74e4cb7cd" containerName="payload-processing"
Sep 11 23:40:44.766537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766502    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9740a1a-23d9-46df-9bcd-28c76ab225db" containerName="payload-pre-processing"
Sep 11 23:40:44.766537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766513    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9740a1a-23d9-46df-9bcd-28c76ab225db" containerName="payload-pre-processing"
Sep 11 23:40:44.766537 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766532    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e" containerName="revoke-keys"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766541    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e" containerName="revoke-keys"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766554    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1da8b65-21d5-42c8-826b-3a3f31169755" containerName="payload-pre-processing"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766563    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1da8b65-21d5-42c8-826b-3a3f31169755" containerName="payload-pre-processing"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766641    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1da8b65-21d5-42c8-826b-3a3f31169755" containerName="payload-pre-processing"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766654    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1519560-2009-43ad-ba17-59b74e4cb7cd" containerName="payload-processing"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766663    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5373c17-b2a5-4e6b-9e9b-13cf2e32ba5e" containerName="revoke-keys"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766675    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a2ad887-2bf1-4008-a61a-f7092ba4885d" containerName="payload-processing"
Sep 11 23:40:44.766831 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.766685    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9740a1a-23d9-46df-9bcd-28c76ab225db" containerName="payload-pre-processing"
Sep 11 23:40:44.771179 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.771159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"
Sep 11 23:40:44.774233 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.774214    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 23:40:44.774327 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.774215    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vrx7m\""
Sep 11 23:40:44.777810 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.777785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"]
Sep 11 23:40:44.819385 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.819336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77304ec5-37f2-4d4f-9541-a7dcb0ac191e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4\" (UID: \"77304ec5-37f2-4d4f-9541-a7dcb0ac191e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"
Sep 11 23:40:44.920462 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.920422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77304ec5-37f2-4d4f-9541-a7dcb0ac191e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4\" (UID: \"77304ec5-37f2-4d4f-9541-a7dcb0ac191e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"
Sep 11 23:40:44.921077 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:44.921058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77304ec5-37f2-4d4f-9541-a7dcb0ac191e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4\" (UID: \"77304ec5-37f2-4d4f-9541-a7dcb0ac191e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"
Sep 11 23:40:45.082194 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:45.082157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"
Sep 11 23:40:45.351554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:45.351460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"]
Sep 11 23:40:45.354481 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:40:45.354452    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77304ec5_37f2_4d4f_9541_a7dcb0ac191e.slice/crio-981868df1519c4ed585435d5afabb8f600ad77db832049cb2681522d0ef8db6f WatchSource:0}: Error finding container 981868df1519c4ed585435d5afabb8f600ad77db832049cb2681522d0ef8db6f: Status 404 returned error can't find the container with id 981868df1519c4ed585435d5afabb8f600ad77db832049cb2681522d0ef8db6f
Sep 11 23:40:45.651150 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:45.651116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:45.714410 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:45.714378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4" event={"ID":"77304ec5-37f2-4d4f-9541-a7dcb0ac191e","Type":"ContainerStarted","Data":"beeb116c8acda4a590fc48fa3841dc64e322718d160da6034e575686cdf72295"}
Sep 11 23:40:45.714554 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:45.714435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4" event={"ID":"77304ec5-37f2-4d4f-9541-a7dcb0ac191e","Type":"ContainerStarted","Data":"981868df1519c4ed585435d5afabb8f600ad77db832049cb2681522d0ef8db6f"}
Sep 11 23:40:45.733588 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:45.733540    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4" podStartSLOduration=1.733522679 podStartE2EDuration="1.733522679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:40:45.732432629 +0000 UTC m=+2360.944403803" watchObservedRunningTime="2026-09-11 23:40:45.733522679 +0000 UTC m=+2360.945494084"
Sep 11 23:40:46.406497 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:46.406464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:46.720207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:46.720119    2580 generic.go:356] "Generic (PLEG): container finished" podID="77304ec5-37f2-4d4f-9541-a7dcb0ac191e" containerID="beeb116c8acda4a590fc48fa3841dc64e322718d160da6034e575686cdf72295" exitCode=0
Sep 11 23:40:46.720207 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:46.720182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4" event={"ID":"77304ec5-37f2-4d4f-9541-a7dcb0ac191e","Type":"ContainerDied","Data":"beeb116c8acda4a590fc48fa3841dc64e322718d160da6034e575686cdf72295"}
Sep 11 23:40:47.852398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:47.852370    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"
Sep 11 23:40:47.948195 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:47.948149    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77304ec5-37f2-4d4f-9541-a7dcb0ac191e-maas-api-service-ca\") pod \"77304ec5-37f2-4d4f-9541-a7dcb0ac191e\" (UID: \"77304ec5-37f2-4d4f-9541-a7dcb0ac191e\") "
Sep 11 23:40:47.948531 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:47.948506    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77304ec5-37f2-4d4f-9541-a7dcb0ac191e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77304ec5-37f2-4d4f-9541-a7dcb0ac191e" (UID: "77304ec5-37f2-4d4f-9541-a7dcb0ac191e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:48.049230 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:48.049157    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77304ec5-37f2-4d4f-9541-a7dcb0ac191e-maas-api-service-ca\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:48.316885 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:48.316858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:48.730279 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:48.730198    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"
Sep 11 23:40:48.730419 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:48.730197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4" event={"ID":"77304ec5-37f2-4d4f-9541-a7dcb0ac191e","Type":"ContainerDied","Data":"981868df1519c4ed585435d5afabb8f600ad77db832049cb2681522d0ef8db6f"}
Sep 11 23:40:48.730419 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:48.730323    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="981868df1519c4ed585435d5afabb8f600ad77db832049cb2681522d0ef8db6f"
Sep 11 23:40:55.379212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.379182    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:55.383879 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.383842    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"]
Sep 11 23:40:55.387689 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.387658    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-da9451-29819490-7hqln"]
Sep 11 23:40:55.415508 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.415474    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="807f6a14-6862-42a9-a0b0-b2b1b8f20643" path="/var/lib/kubelet/pods/807f6a14-6862-42a9-a0b0-b2b1b8f20643/volumes"
Sep 11 23:40:55.415718 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.415704    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"]
Sep 11 23:40:55.415954 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.415902    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc" podUID="2519f6bf-5488-4cbb-8f84-1afbd95af1d3" containerName="payload-processing" containerID="cri-o://83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6" gracePeriod=30
Sep 11 23:40:55.420418 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.420384    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"]
Sep 11 23:40:55.420679 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.420607    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9" podUID="1a4d67a6-d434-4bd6-88d4-581181f3e751" containerName="payload-pre-processing" containerID="cri-o://935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110" gracePeriod=30
Sep 11 23:40:55.458557 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.458516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:55.585856 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.585825    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:55.680002 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.679974    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"]
Sep 11 23:40:55.682646 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.682625    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-da9451-t8qt4"]
Sep 11 23:40:55.687103 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.687077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:55.692615 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.692585    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:40:55.695434 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.695414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:40:55.761626 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.761589    2580 generic.go:356] "Generic (PLEG): container finished" podID="2519f6bf-5488-4cbb-8f84-1afbd95af1d3" containerID="83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6" exitCode=0
Sep 11 23:40:55.761823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.761643    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"
Sep 11 23:40:55.761823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.761675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc" event={"ID":"2519f6bf-5488-4cbb-8f84-1afbd95af1d3","Type":"ContainerDied","Data":"83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6"}
Sep 11 23:40:55.761823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.761727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc" event={"ID":"2519f6bf-5488-4cbb-8f84-1afbd95af1d3","Type":"ContainerDied","Data":"8b4bfe4d9f1bfadad76dc66d518edebc83dd0582f9476ad1233609a28b122a67"}
Sep 11 23:40:55.761823 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.761761    2580 scope.go:117] "RemoveContainer" containerID="83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6"
Sep 11 23:40:55.762897 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.762877    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a4d67a6-d434-4bd6-88d4-581181f3e751" containerID="935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110" exitCode=0
Sep 11 23:40:55.762984 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.762944    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"
Sep 11 23:40:55.762984 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.762960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9" event={"ID":"1a4d67a6-d434-4bd6-88d4-581181f3e751","Type":"ContainerDied","Data":"935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110"}
Sep 11 23:40:55.763068 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.763000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9" event={"ID":"1a4d67a6-d434-4bd6-88d4-581181f3e751","Type":"ContainerDied","Data":"15bbf869015339648c02b2f837723cada3ee9f24ef4c1ba857b49cdba2a680db"}
Sep 11 23:40:55.769906 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.769890    2580 scope.go:117] "RemoveContainer" containerID="83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6"
Sep 11 23:40:55.770306 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:40:55.770288    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6\": container with ID starting with 83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6 not found: ID does not exist" containerID="83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6"
Sep 11 23:40:55.770363 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.770314    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6"} err="failed to get container status \"83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6\": rpc error: code = NotFound desc = could not find container \"83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6\": container with ID starting with 83cc3460ca035f3a7fa8f7859e656fd3eeda36990f1ab734b20a171e46540fd6 not found: ID does not exist"
Sep 11 23:40:55.770363 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.770330    2580 scope.go:117] "RemoveContainer" containerID="935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110"
Sep 11 23:40:55.777568 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.777550    2580 scope.go:117] "RemoveContainer" containerID="935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110"
Sep 11 23:40:55.777829 ip-10-0-138-44 kubenswrapper[2580]: E0911 23:40:55.777812    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110\": container with ID starting with 935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110 not found: ID does not exist" containerID="935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110"
Sep 11 23:40:55.777880 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.777835    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110"} err="failed to get container status \"935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110\": rpc error: code = NotFound desc = could not find container \"935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110\": container with ID starting with 935bd5bd4dfc764041a82440c667d8d55cce49248d6fd3a8ae691f8316350110 not found: ID does not exist"
Sep 11 23:40:55.818307 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.818266    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr7xb\" (UniqueName: \"kubernetes.io/projected/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-kube-api-access-cr7xb\") pod \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") "
Sep 11 23:40:55.818479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.818335    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-plugins-config-volume\") pod \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\" (UID: \"2519f6bf-5488-4cbb-8f84-1afbd95af1d3\") "
Sep 11 23:40:55.818479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.818376    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlxx9\" (UniqueName: \"kubernetes.io/projected/1a4d67a6-d434-4bd6-88d4-581181f3e751-kube-api-access-mlxx9\") pod \"1a4d67a6-d434-4bd6-88d4-581181f3e751\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") "
Sep 11 23:40:55.818479 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.818428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4d67a6-d434-4bd6-88d4-581181f3e751-plugins-config-volume\") pod \"1a4d67a6-d434-4bd6-88d4-581181f3e751\" (UID: \"1a4d67a6-d434-4bd6-88d4-581181f3e751\") "
Sep 11 23:40:55.818784 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.818759    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2519f6bf-5488-4cbb-8f84-1afbd95af1d3" (UID: "2519f6bf-5488-4cbb-8f84-1afbd95af1d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:55.818784 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.818773    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a4d67a6-d434-4bd6-88d4-581181f3e751-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a4d67a6-d434-4bd6-88d4-581181f3e751" (UID: "1a4d67a6-d434-4bd6-88d4-581181f3e751"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:55.820463 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.820442    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4d67a6-d434-4bd6-88d4-581181f3e751-kube-api-access-mlxx9" (OuterVolumeSpecName: "kube-api-access-mlxx9") pod "1a4d67a6-d434-4bd6-88d4-581181f3e751" (UID: "1a4d67a6-d434-4bd6-88d4-581181f3e751"). InnerVolumeSpecName "kube-api-access-mlxx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:55.820530 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.820481    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-kube-api-access-cr7xb" (OuterVolumeSpecName: "kube-api-access-cr7xb") pod "2519f6bf-5488-4cbb-8f84-1afbd95af1d3" (UID: "2519f6bf-5488-4cbb-8f84-1afbd95af1d3"). InnerVolumeSpecName "kube-api-access-cr7xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:55.920063 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.919961    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlxx9\" (UniqueName: \"kubernetes.io/projected/1a4d67a6-d434-4bd6-88d4-581181f3e751-kube-api-access-mlxx9\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:55.920063 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.919997    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a4d67a6-d434-4bd6-88d4-581181f3e751-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:55.920063 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.920010    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr7xb\" (UniqueName: \"kubernetes.io/projected/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-kube-api-access-cr7xb\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:55.920063 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.920021    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2519f6bf-5488-4cbb-8f84-1afbd95af1d3-plugins-config-volume\") on node \"ip-10-0-138-44.ec2.internal\" DevicePath \"\""
Sep 11 23:40:55.979584 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:55.979548    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:56.085808 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:56.085778    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"]
Sep 11 23:40:56.092838 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:56.092813    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-da9451-68db85ddf4-n6gb9"]
Sep 11 23:40:56.104474 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:56.104446    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"]
Sep 11 23:40:56.108548 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:56.108523    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-da9451-6c58f4fd45-5j2rc"]
Sep 11 23:40:56.133435 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:56.133412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:40:57.413056 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:57.413022    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a4d67a6-d434-4bd6-88d4-581181f3e751" path="/var/lib/kubelet/pods/1a4d67a6-d434-4bd6-88d4-581181f3e751/volumes"
Sep 11 23:40:57.413437 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:57.413373    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2519f6bf-5488-4cbb-8f84-1afbd95af1d3" path="/var/lib/kubelet/pods/2519f6bf-5488-4cbb-8f84-1afbd95af1d3/volumes"
Sep 11 23:40:57.413662 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:40:57.413650    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77304ec5-37f2-4d4f-9541-a7dcb0ac191e" path="/var/lib/kubelet/pods/77304ec5-37f2-4d4f-9541-a7dcb0ac191e/volumes"
Sep 11 23:41:01.919867 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:41:01.919827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:41:02.023052 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:41:02.023013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:41:02.180738 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:41:02.180583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:41:25.821363 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:41:25.821328    2580 scope.go:117] "RemoveContainer" containerID="321d4421f36586300a914e435285ef70ba8e9ca786ec5992142d9ac82f603fa1"
Sep 11 23:41:25.829180 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:41:25.829158    2580 scope.go:117] "RemoveContainer" containerID="7f76f67d6ed0593a414bdfdcbc37ace0f9593204374374e34624bd114f11cc69"
Sep 11 23:42:10.005403 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:42:10.005360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:42:11.130799 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:42:11.130767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:42:14.046496 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:42:14.046457    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:42:26.021338 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:42:26.021303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:42:27.688641 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:42:27.688609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:43:06.461611 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:43:06.461571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:43:18.300723 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:43:18.300636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:43:23.744241 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:43:23.744207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:43:54.849735 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:43:54.849698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:43:54.947954 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:43:54.947899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:44:57.375168 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:44:57.375085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:44:57.433707 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:44:57.433643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:45:25.887987 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:45:25.887944    2580 scope.go:117] "RemoveContainer" containerID="33c3beea776b83dd1463b91d3c1a738d9ae90eadffd70fcbee3f647bdeeadd0d"
Sep 11 23:45:29.089441 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:45:29.089404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:45:38.541866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:45:38.541829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:45:55.434755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:45:55.434716    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:46:41.521330 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:46:41.521212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:46:42.675233 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:46:42.675198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:47:25.914863 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:25.914831    2580 scope.go:117] "RemoveContainer" containerID="beeb116c8acda4a590fc48fa3841dc64e322718d160da6034e575686cdf72295"
Sep 11 23:47:35.472270 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:35.472230    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85bdb47898-8c8q6_58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81/authorino/0.log"
Sep 11 23:47:41.138900 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:41.138867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-sftvs_e0139f08-8083-4e56-bd5c-d0a328a9a18f/copy-manifests/0.log"
Sep 11 23:47:41.156719 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:41.156679    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-sftvs_e0139f08-8083-4e56-bd5c-d0a328a9a18f/manager/0.log"
Sep 11 23:47:42.869266 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:42.869223    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85bdb47898-8c8q6_58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81/authorino/0.log"
Sep 11 23:47:43.190658 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:43.190566    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-br7hd_eeebb0b9-3e95-4c85-a6d9-05acb01b84db/kuadrant-console-plugin/0.log"
Sep 11 23:47:44.082662 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:44.082636    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c75bd8f97-b9d4w_2d6d34ee-6d2c-4771-9874-7784fd39dd3f/kube-auth-proxy/0.log"
Sep 11 23:47:44.544390 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:44.544307    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6cdvp_5174e8da-57b9-402d-911d-c03550f89619/payload-processing/0.log"
Sep 11 23:47:44.974572 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:44.974543    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6v4wd_bc7f0911-6140-4e16-9790-a9cddf61b8c4/main/0.log"
Sep 11 23:47:45.187828 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:45.187795    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-zkwp2_c0f28059-ef18-4d76-8eb0-45d0c507d5c3/main/0.log"
Sep 11 23:47:45.393930 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:45.393889    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6npd9_d0e94e10-ccb0-4523-bbec-64bedb676659/main/0.log"
Sep 11 23:47:45.609607 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:45.609561    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-c88np_9ebc3fab-94d0-4293-ae41-6ab5848bd940/main/0.log"
Sep 11 23:47:50.013833 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.013720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:47:50.895460 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895423    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m6z6g/must-gather-2d2qw"]
Sep 11 23:47:50.895728 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895715    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2519f6bf-5488-4cbb-8f84-1afbd95af1d3" containerName="payload-processing"
Sep 11 23:47:50.895781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895729    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2519f6bf-5488-4cbb-8f84-1afbd95af1d3" containerName="payload-processing"
Sep 11 23:47:50.895781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895746    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77304ec5-37f2-4d4f-9541-a7dcb0ac191e" containerName="revoke-keys"
Sep 11 23:47:50.895781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895752    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77304ec5-37f2-4d4f-9541-a7dcb0ac191e" containerName="revoke-keys"
Sep 11 23:47:50.895781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895765    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4d67a6-d434-4bd6-88d4-581181f3e751" containerName="payload-pre-processing"
Sep 11 23:47:50.895781 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895771    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4d67a6-d434-4bd6-88d4-581181f3e751" containerName="payload-pre-processing"
Sep 11 23:47:50.895972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895833    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4d67a6-d434-4bd6-88d4-581181f3e751" containerName="payload-pre-processing"
Sep 11 23:47:50.895972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895841    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2519f6bf-5488-4cbb-8f84-1afbd95af1d3" containerName="payload-processing"
Sep 11 23:47:50.895972 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.895848    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77304ec5-37f2-4d4f-9541-a7dcb0ac191e" containerName="revoke-keys"
Sep 11 23:47:50.898750 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.898733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:50.901472 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.901451    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m6z6g\"/\"default-dockercfg-j5mj9\""
Sep 11 23:47:50.901572 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.901452    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6z6g\"/\"openshift-service-ca.crt\""
Sep 11 23:47:50.902771 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.902755    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6z6g\"/\"kube-root-ca.crt\""
Sep 11 23:47:50.907455 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.907429    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6z6g/must-gather-2d2qw"]
Sep 11 23:47:50.951312 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.951282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7358b0c9-2874-42c4-972c-730995426823-must-gather-output\") pod \"must-gather-2d2qw\" (UID: \"7358b0c9-2874-42c4-972c-730995426823\") " pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:50.951312 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:50.951322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jphx\" (UniqueName: \"kubernetes.io/projected/7358b0c9-2874-42c4-972c-730995426823-kube-api-access-8jphx\") pod \"must-gather-2d2qw\" (UID: \"7358b0c9-2874-42c4-972c-730995426823\") " pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:51.052262 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.052226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jphx\" (UniqueName: \"kubernetes.io/projected/7358b0c9-2874-42c4-972c-730995426823-kube-api-access-8jphx\") pod \"must-gather-2d2qw\" (UID: \"7358b0c9-2874-42c4-972c-730995426823\") " pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:51.052659 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.052313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7358b0c9-2874-42c4-972c-730995426823-must-gather-output\") pod \"must-gather-2d2qw\" (UID: \"7358b0c9-2874-42c4-972c-730995426823\") " pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:51.052659 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.052601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7358b0c9-2874-42c4-972c-730995426823-must-gather-output\") pod \"must-gather-2d2qw\" (UID: \"7358b0c9-2874-42c4-972c-730995426823\") " pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:51.060653 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.060624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jphx\" (UniqueName: \"kubernetes.io/projected/7358b0c9-2874-42c4-972c-730995426823-kube-api-access-8jphx\") pod \"must-gather-2d2qw\" (UID: \"7358b0c9-2874-42c4-972c-730995426823\") " pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:51.208458 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.208366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6z6g/must-gather-2d2qw"
Sep 11 23:47:51.472413 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.472320    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log" limit=1024
Sep 11 23:47:51.477038 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.477010    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6z6g/must-gather-2d2qw"]
Sep 11 23:47:51.480085 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:47:51.480056    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7358b0c9_2874_42c4_972c_730995426823.slice/crio-3f6620180d94a97cf2ea78604fec5b60b4e47d3528ee4c87f5391cf62e3e6d43 WatchSource:0}: Error finding container 3f6620180d94a97cf2ea78604fec5b60b4e47d3528ee4c87f5391cf62e3e6d43: Status 404 returned error can't find the container with id 3f6620180d94a97cf2ea78604fec5b60b4e47d3528ee4c87f5391cf62e3e6d43
Sep 11 23:47:51.481959 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.481938    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:47:51.534567 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:51.534526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6z6g/must-gather-2d2qw" event={"ID":"7358b0c9-2874-42c4-972c-730995426823","Type":"ContainerStarted","Data":"3f6620180d94a97cf2ea78604fec5b60b4e47d3528ee4c87f5391cf62e3e6d43"}
Sep 11 23:47:52.541498 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:52.541454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6z6g/must-gather-2d2qw" event={"ID":"7358b0c9-2874-42c4-972c-730995426823","Type":"ContainerStarted","Data":"291363b9cc5132e02cfa04a19f5a1f2231821dc6a7269895b10a39dec708ddc9"}
Sep 11 23:47:52.541878 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:52.541512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6z6g/must-gather-2d2qw" event={"ID":"7358b0c9-2874-42c4-972c-730995426823","Type":"ContainerStarted","Data":"4f18842252ffe25692e6c6a4981ef6de17e07fb87a5667fef72e0c05bb1e1f0a"}
Sep 11 23:47:52.560000 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:52.559852    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m6z6g/must-gather-2d2qw" podStartSLOduration=1.745485982 podStartE2EDuration="2.559829515s" totalImagesPullingTime="814.343533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:47:52.558226956 +0000 UTC m=+2787.770198131" watchObservedRunningTime="2026-09-11 23:47:52.559829515 +0000 UTC m=+2787.771800690"
Sep 11 23:47:53.914317 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:53.914270    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j2kzp_086cc3c6-5330-4ef2-add4-e8892b85e117/global-pull-secret-syncer/0.log"
Sep 11 23:47:53.973117 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:53.973080    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bnzhf_440d1a51-bd49-49b0-8566-314cf9a9bd17/konnectivity-agent/0.log"
Sep 11 23:47:54.109784 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:54.109755    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-44.ec2.internal_067c5e3fd43aceeec531c7141228c7cb/haproxy/0.log"
Sep 11 23:47:58.497486 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:58.496985    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85bdb47898-8c8q6_58e3ab41-dd59-4d7a-b9f1-25dd13d4cd81/authorino/0.log"
Sep 11 23:47:58.585860 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:47:58.585819    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-br7hd_eeebb0b9-3e95-4c85-a6d9-05acb01b84db/kuadrant-console-plugin/0.log"
Sep 11 23:48:00.558542 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:00.558498    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z59sm_c9353ba7-c74c-497b-b3dd-8092763474d9/node-exporter/0.log"
Sep 11 23:48:00.581398 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:00.581356    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z59sm_c9353ba7-c74c-497b-b3dd-8092763474d9/kube-rbac-proxy/0.log"
Sep 11 23:48:00.607578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:00.607552    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z59sm_c9353ba7-c74c-497b-b3dd-8092763474d9/init-textfile/0.log"
Sep 11 23:48:00.653578 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:00.653060    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8vfbr_d1d6260f-708e-464d-b836-e92c47e9e8b5/kube-rbac-proxy-main/0.log"
Sep 11 23:48:00.688933 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:00.688815    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8vfbr_d1d6260f-708e-464d-b836-e92c47e9e8b5/kube-rbac-proxy-self/0.log"
Sep 11 23:48:00.714872 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:00.714843    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8vfbr_d1d6260f-708e-464d-b836-e92c47e9e8b5/openshift-state-metrics/0.log"
Sep 11 23:48:04.078346 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:04.078312    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-588798ddb-bvhk6_3f84190e-7954-470a-a346-2dd304ecebc6/console/0.log"
Sep 11 23:48:05.005385 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.005346    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"]
Sep 11 23:48:05.008736 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.008709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.025262 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.022730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"]
Sep 11 23:48:05.105203 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.105156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-proc\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.105696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.105240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhlc8\" (UniqueName: \"kubernetes.io/projected/a87e7f32-467a-4064-a850-646d7b85b573-kube-api-access-bhlc8\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.105696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.105281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-sys\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.105696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.105306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-lib-modules\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.105696 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.105359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-podres\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.206769 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.206729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-proc\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207179 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhlc8\" (UniqueName: \"kubernetes.io/projected/a87e7f32-467a-4064-a850-646d7b85b573-kube-api-access-bhlc8\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-sys\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-lib-modules\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-proc\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207309 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-podres\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-sys\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-lib-modules\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.207512 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.207447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a87e7f32-467a-4064-a850-646d7b85b573-podres\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.215547 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.215507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhlc8\" (UniqueName: \"kubernetes.io/projected/a87e7f32-467a-4064-a850-646d7b85b573-kube-api-access-bhlc8\") pod \"perf-node-gather-daemonset-8p9dc\" (UID: \"a87e7f32-467a-4064-a850-646d7b85b573\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.327069 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.327016    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:05.621901 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.621807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"]
Sep 11 23:48:05.629030 ip-10-0-138-44 kubenswrapper[2580]: W0911 23:48:05.629001    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda87e7f32_467a_4064_a850_646d7b85b573.slice/crio-5c5ef357560ac6d61eb6a3f593bf4abe4f2ff63721c119194e4707ffc89d2e2a WatchSource:0}: Error finding container 5c5ef357560ac6d61eb6a3f593bf4abe4f2ff63721c119194e4707ffc89d2e2a: Status 404 returned error can't find the container with id 5c5ef357560ac6d61eb6a3f593bf4abe4f2ff63721c119194e4707ffc89d2e2a
Sep 11 23:48:05.638964 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:05.638926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc" event={"ID":"a87e7f32-467a-4064-a850-646d7b85b573","Type":"ContainerStarted","Data":"5c5ef357560ac6d61eb6a3f593bf4abe4f2ff63721c119194e4707ffc89d2e2a"}
Sep 11 23:48:06.073212 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:06.073181    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-md4vn_9a1cb00f-c928-4518-9867-cbec17193f4d/dns/0.log"
Sep 11 23:48:06.095088 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:06.095057    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-md4vn_9a1cb00f-c928-4518-9867-cbec17193f4d/kube-rbac-proxy/0.log"
Sep 11 23:48:06.116646 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:06.116619    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f9ttb_fea4091a-9169-4cee-b245-2f909063cfb6/dns-node-resolver/0.log"
Sep 11 23:48:06.648866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:06.647980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc" event={"ID":"a87e7f32-467a-4064-a850-646d7b85b573","Type":"ContainerStarted","Data":"b03768e63a0a3f0885de1bd0259e76634529f390650592b77718a9b75a9aa245"}
Sep 11 23:48:06.648866 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:06.648808    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:06.667549 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:06.667485    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc" podStartSLOduration=2.667464349 podStartE2EDuration="2.667464349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:48:06.665299421 +0000 UTC m=+2801.877270594" watchObservedRunningTime="2026-09-11 23:48:06.667464349 +0000 UTC m=+2801.879435524"
Sep 11 23:48:06.708511 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:06.708480    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h2f6t_dcf7d1ff-9c3b-4cc5-ab5b-6d7ad6e965c5/node-ca/0.log"
Sep 11 23:48:08.117073 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:08.117028    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c75bd8f97-b9d4w_2d6d34ee-6d2c-4771-9874-7784fd39dd3f/kube-auth-proxy/0.log"
Sep 11 23:48:08.240801 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:08.240764    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6cdvp_5174e8da-57b9-402d-911d-c03550f89619/payload-processing/0.log"
Sep 11 23:48:09.172189 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:09.172151    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xrmq9_25084dda-89f7-45d8-aced-38d4c22a2ce6/serve-healthcheck-canary/0.log"
Sep 11 23:48:09.707043 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:09.707008    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vs8dz_bd3ca046-9348-4291-a9a9-4ce870b83f00/kube-rbac-proxy/0.log"
Sep 11 23:48:09.725871 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:09.725829    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vs8dz_bd3ca046-9348-4291-a9a9-4ce870b83f00/exporter/0.log"
Sep 11 23:48:09.746127 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:09.746094    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vs8dz_bd3ca046-9348-4291-a9a9-4ce870b83f00/extractor/0.log"
Sep 11 23:48:12.511340 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:12.511294    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-sftvs_e0139f08-8083-4e56-bd5c-d0a328a9a18f/manager/0.log"
Sep 11 23:48:12.528616 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:12.528590    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-sftvs_e0139f08-8083-4e56-bd5c-d0a328a9a18f/copy-manifests/0.log"
Sep 11 23:48:13.668653 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:13.668620    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-8p9dc"
Sep 11 23:48:13.811402 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:13.811370    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5956d5f5bd-qzpz8_221f6034-5d10-43b4-a24b-9504d2b26b42/manager/0.log"
Sep 11 23:48:14.933923 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:14.933894    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c75bd8f97-b9d4w_2d6d34ee-6d2c-4771-9874-7784fd39dd3f/kube-auth-proxy/0.log"
Sep 11 23:48:15.022751 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:15.022657    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6cdvp_5174e8da-57b9-402d-911d-c03550f89619/payload-processing/0.log"
Sep 11 23:48:17.170887 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:17.170844    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c75bd8f97-b9d4w_2d6d34ee-6d2c-4771-9874-7784fd39dd3f/kube-auth-proxy/0.log"
Sep 11 23:48:17.271007 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:17.270971    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-6cdvp_5174e8da-57b9-402d-911d-c03550f89619/payload-processing/0.log"
Sep 11 23:48:20.680056 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:20.680026    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hwws_f174230a-a12c-4a6b-9f61-9c4aab306aa5/kube-multus-additional-cni-plugins/0.log"
Sep 11 23:48:20.699211 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:20.699177    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hwws_f174230a-a12c-4a6b-9f61-9c4aab306aa5/egress-router-binary-copy/0.log"
Sep 11 23:48:20.720025 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:20.719993    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hwws_f174230a-a12c-4a6b-9f61-9c4aab306aa5/cni-plugins/0.log"
Sep 11 23:48:20.740250 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:20.740217    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hwws_f174230a-a12c-4a6b-9f61-9c4aab306aa5/bond-cni-plugin/0.log"
Sep 11 23:48:20.758970 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:20.758943    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hwws_f174230a-a12c-4a6b-9f61-9c4aab306aa5/routeoverride-cni/0.log"
Sep 11 23:48:20.778004 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:20.777972    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hwws_f174230a-a12c-4a6b-9f61-9c4aab306aa5/whereabouts-cni-bincopy/0.log"
Sep 11 23:48:20.799585 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:20.799550    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hwws_f174230a-a12c-4a6b-9f61-9c4aab306aa5/whereabouts-cni/0.log"
Sep 11 23:48:21.181616 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:21.181579    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n8m6b_ab5aa01f-91a4-40dc-8c1e-a18e1697a45f/kube-multus/0.log"
Sep 11 23:48:21.243478 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:21.243446    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9msgs_8dd3f622-28c8-42d7-bd7c-db92655889ba/network-metrics-daemon/0.log"
Sep 11 23:48:21.258323 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:21.258297    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9msgs_8dd3f622-28c8-42d7-bd7c-db92655889ba/kube-rbac-proxy/0.log"
Sep 11 23:48:22.143677 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.143639    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/ovn-controller/0.log"
Sep 11 23:48:22.159294 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.159265    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/ovn-acl-logging/0.log"
Sep 11 23:48:22.185936 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.185877    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/kube-rbac-proxy-node/0.log"
Sep 11 23:48:22.209755 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.209726    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/kube-rbac-proxy-ovn-metrics/0.log"
Sep 11 23:48:22.225297 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.225268    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/northd/0.log"
Sep 11 23:48:22.244321 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.244291    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/nbdb/0.log"
Sep 11 23:48:22.283013 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.282972    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/sbdb/0.log"
Sep 11 23:48:22.374285 ip-10-0-138-44 kubenswrapper[2580]: I0911 23:48:22.374248    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7mn4n_821b3ae3-4605-47ae-a13b-5027a4ac8b9b/ovnkube-controller/0.log"