Sep 16 13:31:00.782814 ip-10-0-130-207 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 13:31:00.782824 ip-10-0-130-207 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 13:31:00.782834 ip-10-0-130-207 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 13:31:00.783060 ip-10-0-130-207 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 13:31:11.032132 ip-10-0-130-207 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 13:31:11.032153 ip-10-0-130-207 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b0ac07cf64f844a99055957f8acb927c --
Sep 16 13:32:52.210182 ip-10-0-130-207 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 13:32:52.574522 ip-10-0-130-207 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 13:32:52.574522 ip-10-0-130-207 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 13:32:52.574522 ip-10-0-130-207 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 13:32:52.574522 ip-10-0-130-207 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 13:32:52.574522 ip-10-0-130-207 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 13:32:52.576261 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.576178    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 13:32:52.579670 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579654    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:32:52.579670 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579670    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579673    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579677    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579681    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579684    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579687    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579690    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579693    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579696    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579699    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579702    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579704    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579708    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579711    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579713    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579716    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579719    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579721    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579724    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579727    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:32:52.579732 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579730    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579732    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579735    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579738    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579741    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579744    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579746    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579749    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579752    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579754    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579757    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579760    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579762    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579765    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579768    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579771    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579775    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579785    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579788    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579790    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:32:52.580229 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579793    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579795    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579797    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579800    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579803    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579805    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579808    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579810    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579812    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579815    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579817    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579820    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579822    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579825    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579828    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579837    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579840    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579843    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579845    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579848    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:32:52.580719 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579851    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579853    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579856    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579860    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579863    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579867    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579870    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579873    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579875    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579878    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579882    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579885    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579887    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579890    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579892    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579895    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579911    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579913    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579916    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579918    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:32:52.581257 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579921    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579923    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579929    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579933    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579936    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579939    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579942    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579945    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579949    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579952    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579955    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579957    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579960    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579963    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579965    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579968    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579970    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.579973    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:32:52.581744 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580701    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580708    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580711    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580715    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580717    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580721    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580724    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580726    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580729    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580732    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580734    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580737    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580740    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580743    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580745    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580748    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580752    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580754    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580757    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580760    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:32:52.582210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580764    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580767    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580769    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580772    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580774    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580777    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580780    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580782    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580784    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580787    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580790    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580792    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580795    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580797    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580799    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580802    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580804    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580807    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580824    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580827    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:32:52.582720 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580829    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580832    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580834    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580837    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580839    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580842    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580844    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580847    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580852    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580856    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580859    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580862    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580865    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580869    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580872    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580875    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580877    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580880    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580882    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580884    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:32:52.583223 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580887    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580889    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580892    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580895    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580914    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580917    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580919    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580922    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580924    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580927    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580929    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580932    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580935    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580939    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580944    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580947    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580950    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580953    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580957    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:32:52.583726 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580960    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580963    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580966    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580969    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580972    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580975    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580977    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580980    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580983    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580985    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580988    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580990    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580993    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580996    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.580998    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581000    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581003    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581005    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581008    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581010    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:32:52.584218 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581386    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581390    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581394    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581397    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581400    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581403    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581406    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581409    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581411    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581414    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581417    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581419    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581422    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581424    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581427    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581429    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581433    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581435    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581438    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581441    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:32:52.584705 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581444    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581446    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581451    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581454    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581457    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581460    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581464    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581467    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581470    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581472    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581475    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581478    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581481    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581484    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581486    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581489    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581491    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581498    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581500    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:32:52.585207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581503    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581505    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581508    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581510    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581513    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581515    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581518    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581520    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581523    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581526    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581529    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581531    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581542    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581544    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581547    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581550    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581552    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581555    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581558    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581560    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:32:52.585676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581563    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581565    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581568    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581571    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581573    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581576    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581578    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581581    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581583    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581586    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581591    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581596    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581600    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581604    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581606    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581609    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581612    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581615    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581618    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:32:52.586210 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581620    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581623    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581626    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581628    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581631    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581634    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581636    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581639    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581642    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581644    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581647    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581649    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581652    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581654    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581657    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581660    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581662    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581665    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581667    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581670    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:32:52.586676 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.581672    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581745    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581781    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581789    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581794    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581806    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581810    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581814    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581818    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581821    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581824    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581828    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581831    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581834    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581837    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581840    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581843    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581846    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581849    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581856    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581858    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581861    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581864    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581867    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 13:32:52.587215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581871    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581874    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581877    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581880    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581883    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581886    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581889    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581892    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581907    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581912    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581915    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581919    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581922    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581927    2580 flags.go:64] FLAG: --enable-server="true"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581935    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581942    2580 flags.go:64] FLAG: --event-burst="100"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581945    2580 flags.go:64] FLAG: --event-qps="50"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581949    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581951    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581954    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581958    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581961    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581964    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581967    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581970    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 13:32:52.587790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581973    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581975    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581979    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581981    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581984    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581987    2580 flags.go:64] FLAG: --feature-gates=""
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581994    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.581997    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582000    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582003    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582006    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582009    2580 flags.go:64] FLAG: --help="false"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582012    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-130-207.ec2.internal"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582015    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582018    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582021    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582025    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582028    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582032    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582035    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582039    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582042    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582051    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582054    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 13:32:52.588417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582057    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582060    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582063    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582066    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582069    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582072    2580 flags.go:64] FLAG: --lock-file=""
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582075    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582078    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582081    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582086    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582089    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582091    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582094    2580 flags.go:64] FLAG: --logging-format="text"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582097    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582100    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582103    2580 flags.go:64] FLAG: --manifest-url=""
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582105    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582109    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582112    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582116    2580 flags.go:64] FLAG: --max-pods="110"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582119    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582122    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582125    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582128    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582130    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 13:32:52.589037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582133    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582137    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582145    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582149    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582152    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582155    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582163    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582169    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582172    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582175    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582178    2580 flags.go:64] FLAG: --port="10250"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582181    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582184    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09c8409ce217840b6"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582187    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582190    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582193    2580 flags.go:64] FLAG: --register-node="true"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582196    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582199    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582202    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582205    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582208    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582212    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582215    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582218    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582221    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 13:32:52.589688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582223    2580 flags.go:64] FLAG: --runonce="false"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582226    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582230    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582233    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582235    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582238    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582241    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582244    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582247    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582251    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582255    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582259    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582261    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582264    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582273    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582276    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582281    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582284    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582287    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582294    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582296    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582299    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582302    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582304    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582307    2580 flags.go:64] FLAG: --v="2"
Sep 16 13:32:52.590312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582311    2580 flags.go:64] FLAG: --version="false"
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582315    2580 flags.go:64] FLAG: --vmodule=""
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582319    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.582322    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582465    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582469    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582472    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582475    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582478    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582481    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582483    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582486    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582489    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582492    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582495    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582497    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582502    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582505    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582508    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582511    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582514    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:32:52.590928 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582517    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582525    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582528    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582530    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582533    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582536    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582538    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582541    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582543    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582546    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582548    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582551    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582554    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582556    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582559    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582561    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582565    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582567    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582570    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582573    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:32:52.591452 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582575    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582578    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582581    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582583    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582586    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582588    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582591    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582595    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582599    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582601    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582604    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582607    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582609    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582613    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582623    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582626    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582628    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582631    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582634    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:32:52.591990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582636    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582639    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582641    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582649    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582652    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582655    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582659    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582662    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582665    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582668    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582671    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582674    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582677    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582679    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582682    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582685    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582688    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582690    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582693    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582696    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:32:52.592465 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582700    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582705    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582708    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582710    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582713    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582716    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582718    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582721    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582738    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582741    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582744    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582746    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582749    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582752    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582754    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582757    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582761    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582764    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582767    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:32:52.592989 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582770    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582773    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582776    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.582779    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.583324    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.590661    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.590674    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.591386    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.591396    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591462    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591467    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591470    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591473    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591477    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:32:52.593482 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591481    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591484    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591487    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591490    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591494    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591497    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591500    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591503    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591506    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591509    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591512    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591515    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591518    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591521    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591524    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591527    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591530    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591533    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591536    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591538    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:32:52.593852 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591541    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591543    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591546    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591548    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591551    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591554    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591556    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591559    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591561    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591564    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591566    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591569    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591572    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591575    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591577    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591580    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591583    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591585    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591588    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591591    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:32:52.594353 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591594    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591596    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591599    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591601    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591604    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591607    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591610    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591612    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591615    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591617    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591620    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591623    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591625    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591628    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591630    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591633    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591635    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591638    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591641    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591643    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:32:52.594851 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591646    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591649    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591651    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591654    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591657    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591660    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591663    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591665    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591669    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591673    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591676    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591678    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591681    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591684    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591687    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591689    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591692    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591695    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591697    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591700    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:32:52.595335 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591703    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591705    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591708    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591710    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591713    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591716    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591718    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591721    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591725    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591729    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591733    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591737    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591740    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591743    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.591748    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 13:32:52.595815 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591871    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591876    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591879    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591883    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591886    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591889    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591891    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591894    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591908    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591911    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591914    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591917    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591920    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591923    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591926    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591928    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591931    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591933    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591936    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591939    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 13:32:52.596207 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591941    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591944    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591946    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591949    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591952    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591954    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591958    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591962    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591964    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591967    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591970    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591973    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591976    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591978    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591981    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591984    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591986    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591988    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591991    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 13:32:52.596764 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591994    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591997    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.591999    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592002    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592006    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592010    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592013    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592016    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592019    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592021    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592024    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592027    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592030    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592032    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592035    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592038    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592040    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592043    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592046    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 13:32:52.597304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592048    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592051    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592054    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592056    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592059    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592062    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592064    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592067    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592070    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592072    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592074    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592077    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592080    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592082    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592085    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592089    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592092    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592094    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592097    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592100    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 13:32:52.597774 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592102    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592105    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592107    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592110    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592112    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592115    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592117    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592119    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592124    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592129    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592131    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592134    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592136    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592139    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592142    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592144    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592147    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592149    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592152    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592154    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 13:32:52.598276 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:52.592157    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 13:32:52.598789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.592162    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 13:32:52.598789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.592257    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 13:32:52.598789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.594176    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 13:32:52.598789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.595134    2580 server.go:1019] "Starting client certificate rotation"
Sep 16 13:32:52.598789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.595230    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 13:32:52.598789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.595265    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 13:32:52.598789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.597934    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 13:32:52.603945 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.603919    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 13:32:52.612279 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.612261    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 16 13:32:52.615845 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.615829    2580 log.go:25] "Validated CRI v1 image API"
Sep 16 13:32:52.616993 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.616970    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 13:32:52.619539 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.619509    2580 fs.go:135] Filesystem UUIDs: map[0f217f34-6f52-460e-a7d3-089bbd1f004b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8245e442-1abb-470b-9e5b-601124aac092:/dev/nvme0n1p4]
Sep 16 13:32:52.619604 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.619535    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 13:32:52.624326 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.624221    2580 manager.go:216] Machine: {Timestamp:2026-09-16 13:32:52.623115476 +0000 UTC m=+0.306773861 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100011 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ac2004994af3238b41f9444222b2a SystemUUID:ec2ac200-4994-af32-38b4-1f9444222b2a BootID:b0ac07cf-64f8-44a9-9055-957f8acb927c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e1:c2:95:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e1:c2:95:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:df:2c:42:12:20 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 13:32:52.624326 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.624322    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 13:32:52.624436 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.624395    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 13:32:52.627354 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.627326    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 13:32:52.627493 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.627358    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-207.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 13:32:52.628336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.628326    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 13:32:52.628368 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.628339    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 13:32:52.628368 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.628347    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 13:32:52.629073 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.629062    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 13:32:52.629293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.629278    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 13:32:52.629864 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.629852    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 13:32:52.633640 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.633628    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 13:32:52.633756 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.633748    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 13:32:52.633789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.633775    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 13:32:52.633789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.633785    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 13:32:52.633840 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.633799    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 13:32:52.633840 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.633809    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 16 13:32:52.633840 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.633817    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 13:32:52.634155 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.634140    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 13:32:52.634155 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.634159    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 13:32:52.636274 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.636259    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 13:32:52.638265 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.638250    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 13:32:52.638343 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.638288    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 13:32:52.639709 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639697    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639715    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639732    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639741    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639753    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639762    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639782    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639790    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 13:32:52.639800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639800    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 13:32:52.640059 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639808    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 13:32:52.640059 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639826    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 13:32:52.640059 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.639835    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 13:32:52.641105 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.641093    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 13:32:52.641182 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.641112    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 13:32:52.643355 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.643329    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 13:32:52.643442 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.643362    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 13:32:52.643710 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.643695    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-207.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 13:32:52.644721 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.644705    2580 server.go:1268] "Started kubelet"
Sep 16 13:32:52.644824 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.644794    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 13:32:52.644870 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.644800    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 13:32:52.644933 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.644872    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 13:32:52.648894 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.648870    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 13:32:52.649058 ip-10-0-130-207 systemd[1]: Started Kubernetes Kubelet.
Sep 16 13:32:52.650101 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.649734    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 13:32:52.650468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.650425    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 13:32:52.653741 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.652763    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-207.ec2.internal.18d5d06823c1399e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-207.ec2.internal,UID:ip-10-0-130-207.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-207.ec2.internal,},FirstTimestamp:2026-09-16 13:32:52.644682142 +0000 UTC m=+0.328340527,LastTimestamp:2026-09-16 13:32:52.644682142 +0000 UTC m=+0.328340527,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-207.ec2.internal,}"
Sep 16 13:32:52.658943 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.658921    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 13:32:52.659532 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.659517    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 13:32:52.659807 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.659788    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:52.660271 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660252    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 13:32:52.660271 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660256    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 13:32:52.660412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660291    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 13:32:52.660412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660388    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 13:32:52.660412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660395    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 13:32:52.660767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660748    2580 factory.go:153] Registering CRI-O factory
Sep 16 13:32:52.660833 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660772    2580 factory.go:223] Registration of the crio container factory successfully
Sep 16 13:32:52.660885 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660837    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 13:32:52.660885 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660848    2580 factory.go:55] Registering systemd factory
Sep 16 13:32:52.660885 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660857    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 16 13:32:52.660885 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660872    2580 factory.go:103] Registering Raw factory
Sep 16 13:32:52.660885 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.660885    2580 manager.go:1196] Started watching for new ooms in manager
Sep 16 13:32:52.661275 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.661263    2580 manager.go:349] Starting recovery of all containers
Sep 16 13:32:52.670669 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.670641    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 13:32:52.670808 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.670783    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-207.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 13:32:52.672343 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.672197    2580 manager.go:354] Recovery completed
Sep 16 13:32:52.673718 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.673634    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kz95p"
Sep 16 13:32:52.675057 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.673994    2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 16 13:32:52.678079 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.678067    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:32:52.679991 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.679974    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:32:52.680066 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.680006    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:32:52.680066 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.680021    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:32:52.680619 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.680603    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 13:32:52.680619 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.680618    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 13:32:52.680619 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.680620    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:32:52.680812 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.680633    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 13:32:52.681596 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.681581    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kz95p"
Sep 16 13:32:52.681774 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.681701    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-207.ec2.internal.18d5d06825dbfc0a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-207.ec2.internal,UID:ip-10-0-130-207.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-207.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-207.ec2.internal,},FirstTimestamp:2026-09-16 13:32:52.679990282 +0000 UTC m=+0.363648672,LastTimestamp:2026-09-16 13:32:52.679990282 +0000 UTC m=+0.363648672,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-207.ec2.internal,}"
Sep 16 13:32:52.682411 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.682393    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:32:52.682496 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.682423    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:32:52.682496 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.682436    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:32:52.682824 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.682801    2580 policy_none.go:49] "None policy: Start"
Sep 16 13:32:52.682824 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.682826    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 13:32:52.682963 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.682840    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 13:32:52.684778 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.684763    2580 policy_none.go:47] "Start"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.717426    2580 manager.go:337] "Starting Device Plugin manager"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.717458    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.717467    2580 server.go:87] "Starting device plugin registration server"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.717748    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.717761    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.717856    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.717955    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.717963    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.718529    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 13:32:52.737197 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.718582    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:52.795004 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.794962    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 13:32:52.796150 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.796133    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 13:32:52.796246 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.796178    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 13:32:52.796246 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.796202    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 13:32:52.796333 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.796290    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 13:32:52.798608 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.798586    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:32:52.818619 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.818587    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:32:52.819511 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.819492    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:32:52.819605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.819531    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:32:52.819605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.819543    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:32:52.819605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.819564    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-207.ec2.internal"
Sep 16 13:32:52.828125 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.828062    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-207.ec2.internal"
Sep 16 13:32:52.828125 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.828082    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-207.ec2.internal\": node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:52.852539 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.852518    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:52.896761 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.896733    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"]
Sep 16 13:32:52.896889 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.896799    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:32:52.898323 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.898303    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:32:52.898420 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.898335    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:32:52.898420 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.898347    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:32:52.900501 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.900487    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:32:52.900631 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.900615    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:52.900688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.900653    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:32:52.901247 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.901229    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:32:52.901336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.901260    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:32:52.901336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.901269    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:32:52.901336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.901285    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:32:52.901336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.901307    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:32:52.901336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.901320    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:32:52.903518 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.903505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:52.903572 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.903530    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 13:32:52.904275 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.904253    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 13:32:52.904324 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.904283    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 13:32:52.904324 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:52.904297    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeHasSufficientPID"
Sep 16 13:32:52.928484 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.928468    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-207.ec2.internal\" not found" node="ip-10-0-130-207.ec2.internal"
Sep 16 13:32:52.932883 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.932869    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-207.ec2.internal\" not found" node="ip-10-0-130-207.ec2.internal"
Sep 16 13:32:52.952889 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:52.952868    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.053559 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.053532    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.061810 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.061792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.061896 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.061815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.061896 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.061833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4054c0090de8f54aed802a5d555be580-config\") pod \"kube-apiserver-proxy-ip-10-0-130-207.ec2.internal\" (UID: \"4054c0090de8f54aed802a5d555be580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.154297 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.154226    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.162579 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.162555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.162630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.162589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.162630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.162608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4054c0090de8f54aed802a5d555be580-config\") pod \"kube-apiserver-proxy-ip-10-0-130-207.ec2.internal\" (UID: \"4054c0090de8f54aed802a5d555be580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.162709 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.162691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.162743 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.162723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4054c0090de8f54aed802a5d555be580-config\") pod \"kube-apiserver-proxy-ip-10-0-130-207.ec2.internal\" (UID: \"4054c0090de8f54aed802a5d555be580\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.162781 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.162769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6be15198fb113e0b9a0291128436e59-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal\" (UID: \"a6be15198fb113e0b9a0291128436e59\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.231730 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.231702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.235248 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.235222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.254765 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.254738    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.356618 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.356573    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.462672 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.462574    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.563745 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.563703    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.597367 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.597346    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 13:32:53.597753 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.597511    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 13:32:53.664388 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.664361    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.687083 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.687047    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 13:27:53 +0000 UTC" deadline="2028-03-11 03:03:06.09704446 +0000 UTC"
Sep 16 13:32:53.687083 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.687080    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12997h30m12.409968518s"
Sep 16 13:32:53.765465 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:53.765376    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-207.ec2.internal\" not found"
Sep 16 13:32:53.817210 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.817182    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:32:53.863166 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.863140    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.875467 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.875442    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 13:32:53.877506 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.877492    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal"
Sep 16 13:32:53.885751 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:53.885737    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 13:32:54.075109 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.075040    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:32:54.100120 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:54.100082    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4054c0090de8f54aed802a5d555be580.slice/crio-b17b5897ed7c2b309b043a04b96bb7ae15418451e71ffa6501ce9e663d4dee23 WatchSource:0}: Error finding container b17b5897ed7c2b309b043a04b96bb7ae15418451e71ffa6501ce9e663d4dee23: Status 404 returned error can't find the container with id b17b5897ed7c2b309b043a04b96bb7ae15418451e71ffa6501ce9e663d4dee23
Sep 16 13:32:54.105032 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.105013    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:32:54.117551 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.117522    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:32:54.587420 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.587392    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 13:32:54.636815 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.636777    2580 apiserver.go:52] "Watching apiserver"
Sep 16 13:32:54.643242 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.643215    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 13:32:54.643597 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.643573    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-5655w","openshift-ovn-kubernetes/ovnkube-node-5vh56","kube-system/konnectivity-agent-vj42r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc","openshift-dns/node-resolver-vcjzc","openshift-image-registry/node-ca-h4jzc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal","openshift-multus/multus-4k6cv","openshift-multus/network-metrics-daemon-wszjx","openshift-network-operator/iptables-alerter-kf6d5","kube-system/global-pull-secret-syncer-sxq2j","kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal","openshift-cluster-node-tuning-operator/tuned-l6pz6","openshift-multus/multus-additional-cni-plugins-jlm7v"]
Sep 16 13:32:54.647657 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.647641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.649957 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.649932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.652206 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.652188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.652579 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.652498    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hhwv6\""
Sep 16 13:32:54.652689 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.652631    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 13:32:54.652807 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.652724    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:32:54.652807 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.652785    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 13:32:54.652807 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.652804    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 13:32:54.653026 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.652878    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 13:32:54.653070 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.653041    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 13:32:54.653070 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.653054    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k9j9w\""
Sep 16 13:32:54.653183 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.653109    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 13:32:54.653290 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.653274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 13:32:54.654762 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.654742    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 13:32:54.654896 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.654877    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-59qss\""
Sep 16 13:32:54.654968 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.654891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.655751 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.655577    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 13:32:54.657239 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.657221    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 13:32:54.657363 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.657296    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 13:32:54.657363 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.657342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.657490 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.657302    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8hzzp\""
Sep 16 13:32:54.657591 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.657518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 13:32:54.659365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.659347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 13:32:54.659471 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.659388    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 13:32:54.659471 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.659421    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s69xn\""
Sep 16 13:32:54.659803 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.659786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.660101 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.660078    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 13:32:54.661645 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.661625    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 13:32:54.661747 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.661685    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 13:32:54.661747 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.661691    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 13:32:54.661747 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.661625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mgtx9\""
Sep 16 13:32:54.662291 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.662275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:54.662377 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.662354    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:32:54.664619 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.664602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.667530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.667043    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7v6qh\""
Sep 16 13:32:54.667530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.667130    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:32:54.667530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.667323    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 13:32:54.668735 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.668715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.670733 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.670702    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dptsm\""
Sep 16 13:32:54.670733 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.670709    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 13:32:54.670919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.670750    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 13:32:54.670919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.670841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmm4h\" (UniqueName: \"kubernetes.io/projected/6ea9a2a4-11ee-4027-affe-da9400ccd248-kube-api-access-wmm4h\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:54.670919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.670878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvgl7\" (UniqueName: \"kubernetes.io/projected/90074422-7839-4850-af0a-7c2d48280472-kube-api-access-vvgl7\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.671062 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.670922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snrzb\" (UniqueName: \"kubernetes.io/projected/46a9e2e1-e39a-490d-a9ad-e919b33157dd-kube-api-access-snrzb\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.671062 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.670983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.671062 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed95382d-1fdc-40dd-a557-b20b2e4cb038-host\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.671221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.671221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671106    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 13:32:54.671221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671121    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 13:32:54.671221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br2fw\" (UniqueName: \"kubernetes.io/projected/b83024eb-d98d-4dc6-8e35-81d015973261-kube-api-access-br2fw\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.671221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-device-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.671221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b83024eb-d98d-4dc6-8e35-81d015973261-host-slash\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-etc-selinux\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671144    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671246    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b83024eb-d98d-4dc6-8e35-81d015973261-iptables-alerter-script\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv2gg\" (UniqueName: \"kubernetes.io/projected/6a28437e-71de-45c2-8064-02e2d89aa324-kube-api-access-vv2gg\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4280dfa4-c6ca-4824-8d64-e4d90bf2f176-agent-certs\") pod \"konnectivity-agent-vj42r\" (UID: \"4280dfa4-c6ca-4824-8d64-e4d90bf2f176\") " pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-registration-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a28437e-71de-45c2-8064-02e2d89aa324-hosts-file\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671437    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-system-cni-dir\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.671503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cnibin\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.671522    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdwkb\" (UniqueName: \"kubernetes.io/projected/ed95382d-1fdc-40dd-a557-b20b2e4cb038-kube-api-access-pdwkb\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-socket-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a28437e-71de-45c2-8064-02e2d89aa324-tmp-dir\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed95382d-1fdc-40dd-a557-b20b2e4cb038-serviceca\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4280dfa4-c6ca-4824-8d64-e4d90bf2f176-konnectivity-ca\") pod \"konnectivity-agent-vj42r\" (UID: \"4280dfa4-c6ca-4824-8d64-e4d90bf2f176\") " pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-sys-fs\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.672255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.671854    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 13:32:54.673761 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.673735    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.673828 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.673804    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:32:54.676341 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.676321    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.678276 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.678255    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 13:32:54.678368 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.678344    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t8ll9\""
Sep 16 13:32:54.697790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.697771    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4nqd2"
Sep 16 13:32:54.705787 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.705768    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4nqd2"
Sep 16 13:32:54.761191 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.761166    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 13:32:54.772184 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br2fw\" (UniqueName: \"kubernetes.io/projected/b83024eb-d98d-4dc6-8e35-81d015973261-kube-api-access-br2fw\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.772459 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-device-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.772571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-systemd-units\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.772571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-systemd\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.772693 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-var-lib-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.772693 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovn-node-metrics-cert\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.772693 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b83024eb-d98d-4dc6-8e35-81d015973261-host-slash\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.772693 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-device-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.772880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-etc-selinux\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.772880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-cni-bin\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.772880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b83024eb-d98d-4dc6-8e35-81d015973261-iptables-alerter-script\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.772880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772793    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b83024eb-d98d-4dc6-8e35-81d015973261-host-slash\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.772880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv2gg\" (UniqueName: \"kubernetes.io/projected/6a28437e-71de-45c2-8064-02e2d89aa324-kube-api-access-vv2gg\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.773144 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.773144 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.772988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-registration-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.773144 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-kubelet\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.773281 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-registration-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.773281 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-socket-dir-parent\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.773369 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.773369 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-etc-selinux\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.773587 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b83024eb-d98d-4dc6-8e35-81d015973261-iptables-alerter-script\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.773785 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysctl-d\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.773853 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-run-netns\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.773853 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-etc-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.773964 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-system-cni-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.773964 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtnvd\" (UniqueName: \"kubernetes.io/projected/75292479-8bee-4bc8-8263-e77513e1949b-kube-api-access-dtnvd\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.773964 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-modprobe-d\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.774103 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.773987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a28437e-71de-45c2-8064-02e2d89aa324-hosts-file\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.774412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdwkb\" (UniqueName: \"kubernetes.io/projected/ed95382d-1fdc-40dd-a557-b20b2e4cb038-kube-api-access-pdwkb\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.774485 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6a28437e-71de-45c2-8064-02e2d89aa324-hosts-file\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.774485 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-socket-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.774578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed95382d-1fdc-40dd-a557-b20b2e4cb038-serviceca\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.774578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4280dfa4-c6ca-4824-8d64-e4d90bf2f176-konnectivity-ca\") pod \"konnectivity-agent-vj42r\" (UID: \"4280dfa4-c6ca-4824-8d64-e4d90bf2f176\") " pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.774578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-sys-fs\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.774578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/78390bc8-31f3-4f46-81cb-08992be7587b-kubelet-config\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-socket-dir\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/78390bc8-31f3-4f46-81cb-08992be7587b-dbus\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-cni-bin\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvgl7\" (UniqueName: \"kubernetes.io/projected/90074422-7839-4850-af0a-7c2d48280472-kube-api-access-vvgl7\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90074422-7839-4850-af0a-7c2d48280472-sys-fs\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-env-overrides\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75292479-8bee-4bc8-8263-e77513e1949b-cni-binary-copy\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.774828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-k8s-cni-cncf-io\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-kubernetes\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-slash\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xrfk\" (UniqueName: \"kubernetes.io/projected/50365d01-e172-4f8f-bd73-3f62082b0cf1-kube-api-access-9xrfk\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.774983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/75292479-8bee-4bc8-8263-e77513e1949b-multus-daemon-config\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-systemd\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-var-lib-kubelet\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-netns\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-cni-multus\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4280dfa4-c6ca-4824-8d64-e4d90bf2f176-konnectivity-ca\") pod \"konnectivity-agent-vj42r\" (UID: \"4280dfa4-c6ca-4824-8d64-e4d90bf2f176\") " pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-cni-netd\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-run\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-tmp\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4280dfa4-c6ca-4824-8d64-e4d90bf2f176-agent-certs\") pod \"konnectivity-agent-vj42r\" (UID: \"4280dfa4-c6ca-4824-8d64-e4d90bf2f176\") " pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovnkube-config\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.775322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovnkube-script-lib\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-multus-certs\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-log-socket\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-cnibin\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-kubelet\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysconfig\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-host\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sclqh\" (UniqueName: \"kubernetes.io/projected/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-kube-api-access-sclqh\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ed95382d-1fdc-40dd-a557-b20b2e4cb038-serviceca\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-system-cni-dir\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.775595    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cnibin\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-system-cni-dir\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-ovn\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775692    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 13:32:54.776163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cnibin\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.775722    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:32:55.275672612 +0000 UTC m=+2.959331005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a28437e-71de-45c2-8064-02e2d89aa324-tmp-dir\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmm4h\" (UniqueName: \"kubernetes.io/projected/6ea9a2a4-11ee-4027-affe-da9400ccd248-kube-api-access-wmm4h\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-node-log\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-cni-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-conf-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-etc-kubernetes\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysctl-conf\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.775997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-lib-modules\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snrzb\" (UniqueName: \"kubernetes.io/projected/46a9e2e1-e39a-490d-a9ad-e919b33157dd-kube-api-access-snrzb\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed95382d-1fdc-40dd-a557-b20b2e4cb038-host\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a28437e-71de-45c2-8064-02e2d89aa324-tmp-dir\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-sys\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.777016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776152    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed95382d-1fdc-40dd-a557-b20b2e4cb038-host\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.777531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.777531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.777531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.777531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46a9e2e1-e39a-490d-a9ad-e919b33157dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.777531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-hostroot\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.777531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-tuned\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.777531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.776696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46a9e2e1-e39a-490d-a9ad-e919b33157dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.779381 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.779361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4280dfa4-c6ca-4824-8d64-e4d90bf2f176-agent-certs\") pod \"konnectivity-agent-vj42r\" (UID: \"4280dfa4-c6ca-4824-8d64-e4d90bf2f176\") " pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.782680 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.782651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br2fw\" (UniqueName: \"kubernetes.io/projected/b83024eb-d98d-4dc6-8e35-81d015973261-kube-api-access-br2fw\") pod \"iptables-alerter-kf6d5\" (UID: \"b83024eb-d98d-4dc6-8e35-81d015973261\") " pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.787373 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.787341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdwkb\" (UniqueName: \"kubernetes.io/projected/ed95382d-1fdc-40dd-a557-b20b2e4cb038-kube-api-access-pdwkb\") pod \"node-ca-h4jzc\" (UID: \"ed95382d-1fdc-40dd-a557-b20b2e4cb038\") " pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.787487 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.787420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvgl7\" (UniqueName: \"kubernetes.io/projected/90074422-7839-4850-af0a-7c2d48280472-kube-api-access-vvgl7\") pod \"aws-ebs-csi-driver-node-pchtc\" (UID: \"90074422-7839-4850-af0a-7c2d48280472\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.787682 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.787662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmm4h\" (UniqueName: \"kubernetes.io/projected/6ea9a2a4-11ee-4027-affe-da9400ccd248-kube-api-access-wmm4h\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:54.788113 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.788097    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv2gg\" (UniqueName: \"kubernetes.io/projected/6a28437e-71de-45c2-8064-02e2d89aa324-kube-api-access-vv2gg\") pod \"node-resolver-vcjzc\" (UID: \"6a28437e-71de-45c2-8064-02e2d89aa324\") " pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.788346 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.788326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snrzb\" (UniqueName: \"kubernetes.io/projected/46a9e2e1-e39a-490d-a9ad-e919b33157dd-kube-api-access-snrzb\") pod \"multus-additional-cni-plugins-jlm7v\" (UID: \"46a9e2e1-e39a-490d-a9ad-e919b33157dd\") " pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.800567 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.800524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" event={"ID":"a6be15198fb113e0b9a0291128436e59","Type":"ContainerStarted","Data":"dd28e71021585b905dd49067b6aec61ac55afa7e9677df5b329a3f215617b149"}
Sep 16 13:32:54.801572 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.801551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal" event={"ID":"4054c0090de8f54aed802a5d555be580","Type":"ContainerStarted","Data":"b17b5897ed7c2b309b043a04b96bb7ae15418451e71ffa6501ce9e663d4dee23"}
Sep 16 13:32:54.876966 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.876870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.876966 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.876929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/78390bc8-31f3-4f46-81cb-08992be7587b-kubelet-config\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.876966 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.876947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/78390bc8-31f3-4f46-81cb-08992be7587b-dbus\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.876974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.876995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-cni-bin\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/78390bc8-31f3-4f46-81cb-08992be7587b-kubelet-config\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-env-overrides\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-cni-bin\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877082    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/78390bc8-31f3-4f46-81cb-08992be7587b-dbus\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.876992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.877331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75292479-8bee-4bc8-8263-e77513e1949b-cni-binary-copy\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-k8s-cni-cncf-io\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-kubernetes\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-slash\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-k8s-cni-cncf-io\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-slash\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xrfk\" (UniqueName: \"kubernetes.io/projected/50365d01-e172-4f8f-bd73-3f62082b0cf1-kube-api-access-9xrfk\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-env-overrides\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-kubernetes\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/75292479-8bee-4bc8-8263-e77513e1949b-multus-daemon-config\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.877614    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.877677    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret podName:78390bc8-31f3-4f46-81cb-08992be7587b nodeName:}" failed. No retries permitted until 2026-09-16 13:32:55.377656622 +0000 UTC m=+3.061315003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret") pod "global-pull-secret-syncer-sxq2j" (UID: "78390bc8-31f3-4f46-81cb-08992be7587b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:54.877745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-systemd\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-var-lib-kubelet\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75292479-8bee-4bc8-8263-e77513e1949b-cni-binary-copy\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-systemd\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-netns\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-cni-multus\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-netns\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-var-lib-kubelet\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-cni-netd\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-cni-netd\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877967    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-cni-multus\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.877983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-run\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-tmp\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovnkube-config\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovnkube-script-lib\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-run\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.878443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-multus-certs\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-log-socket\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-cnibin\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-kubelet\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysconfig\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-host\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sclqh\" (UniqueName: \"kubernetes.io/projected/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-kube-api-access-sclqh\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-ovn\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-node-log\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-cnibin\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-cni-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-conf-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/75292479-8bee-4bc8-8263-e77513e1949b-multus-daemon-config\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-etc-kubernetes\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-log-socket\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysctl-conf\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-node-log\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-lib-modules\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.879447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-var-lib-kubelet\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-sys\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovnkube-config\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysconfig\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-host-run-multus-certs\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-conf-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-etc-kubernetes\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovnkube-script-lib\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysctl-conf\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.878867    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-host\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-hostroot\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-ovn\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-tuned\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-hostroot\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-cni-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-sys\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.880398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-systemd-units\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879224    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-lib-modules\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-systemd\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-var-lib-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-systemd-units\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovn-node-metrics-cert\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-run-systemd\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-cni-bin\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-var-lib-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-kubelet\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-kubelet\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-socket-dir-parent\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysctl-d\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-run-netns\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-etc-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-multus-socket-dir-parent\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-cni-bin\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-system-cni-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.881327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtnvd\" (UniqueName: \"kubernetes.io/projected/75292479-8bee-4bc8-8263-e77513e1949b-kube-api-access-dtnvd\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75292479-8bee-4bc8-8263-e77513e1949b-system-cni-dir\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-modprobe-d\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-host-run-netns\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50365d01-e172-4f8f-bd73-3f62082b0cf1-etc-openvswitch\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-modprobe-d\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.879750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-sysctl-d\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.881341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-tmp\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.881776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/50365d01-e172-4f8f-bd73-3f62082b0cf1-ovn-node-metrics-cert\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.882255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.881931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-etc-tuned\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.887129 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.887106    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:32:54.887129 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.887132    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:32:54.887308 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.887147    2580 projected.go:196] Error preparing data for projected volume kube-api-access-974v4 for pod openshift-network-diagnostics/network-check-target-5655w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:54.887308 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:54.887217    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4 podName:3215ed35-059b-46bb-b6ba-15c279a68841 nodeName:}" failed. No retries permitted until 2026-09-16 13:32:55.387198738 +0000 UTC m=+3.070857126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-974v4" (UniqueName: "kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4") pod "network-check-target-5655w" (UID: "3215ed35-059b-46bb-b6ba-15c279a68841") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:54.888639 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.888616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtnvd\" (UniqueName: \"kubernetes.io/projected/75292479-8bee-4bc8-8263-e77513e1949b-kube-api-access-dtnvd\") pod \"multus-4k6cv\" (UID: \"75292479-8bee-4bc8-8263-e77513e1949b\") " pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:54.889004 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.888981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sclqh\" (UniqueName: \"kubernetes.io/projected/75ba721d-4a94-4ae0-89b6-a31d4d62ee3c-kube-api-access-sclqh\") pod \"tuned-l6pz6\" (UID: \"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c\") " pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:54.889674 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.889649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xrfk\" (UniqueName: \"kubernetes.io/projected/50365d01-e172-4f8f-bd73-3f62082b0cf1-kube-api-access-9xrfk\") pod \"ovnkube-node-5vh56\" (UID: \"50365d01-e172-4f8f-bd73-3f62082b0cf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:54.960172 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.960136    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kf6d5"
Sep 16 13:32:54.967739 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.967705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jlm7v"
Sep 16 13:32:54.976278 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.976253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:32:54.981082 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.980974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc"
Sep 16 13:32:54.986515 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.986309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vcjzc"
Sep 16 13:32:54.988220 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:54.988192    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a9e2e1_e39a_490d_a9ad_e919b33157dd.slice/crio-e2b64c31b9266109207ea444e3c33044a2b8bb984bc69ed84e2180393d08b952 WatchSource:0}: Error finding container e2b64c31b9266109207ea444e3c33044a2b8bb984bc69ed84e2180393d08b952: Status 404 returned error can't find the container with id e2b64c31b9266109207ea444e3c33044a2b8bb984bc69ed84e2180393d08b952
Sep 16 13:32:54.992374 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.992349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h4jzc"
Sep 16 13:32:54.994439 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:54.994360    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4280dfa4_c6ca_4824_8d64_e4d90bf2f176.slice/crio-6023f1dae31ef35f964cbd9a57b11e94fd70d0a8cae84d4c33bedda45cac7f61 WatchSource:0}: Error finding container 6023f1dae31ef35f964cbd9a57b11e94fd70d0a8cae84d4c33bedda45cac7f61: Status 404 returned error can't find the container with id 6023f1dae31ef35f964cbd9a57b11e94fd70d0a8cae84d4c33bedda45cac7f61
Sep 16 13:32:54.997397 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:54.997376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l6pz6"
Sep 16 13:32:55.002184 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:55.002153    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a28437e_71de_45c2_8064_02e2d89aa324.slice/crio-72a723317f49893295f2690774868c43af1a410f5ce16efb181cb9b5c87380ef WatchSource:0}: Error finding container 72a723317f49893295f2690774868c43af1a410f5ce16efb181cb9b5c87380ef: Status 404 returned error can't find the container with id 72a723317f49893295f2690774868c43af1a410f5ce16efb181cb9b5c87380ef
Sep 16 13:32:55.003618 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.003595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:32:55.006512 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:55.006484    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded95382d_1fdc_40dd_a557_b20b2e4cb038.slice/crio-e572fed6e97ce244015c403476f60879117348bc8627bb5373493c0394c455f9 WatchSource:0}: Error finding container e572fed6e97ce244015c403476f60879117348bc8627bb5373493c0394c455f9: Status 404 returned error can't find the container with id e572fed6e97ce244015c403476f60879117348bc8627bb5373493c0394c455f9
Sep 16 13:32:55.008620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.008587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k6cv"
Sep 16 13:32:55.012446 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:55.012422    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ba721d_4a94_4ae0_89b6_a31d4d62ee3c.slice/crio-0001879dffa1c15fa6ab82f0ba7195ce7631560e3042fc2c3cdad54e04ccb3cc WatchSource:0}: Error finding container 0001879dffa1c15fa6ab82f0ba7195ce7631560e3042fc2c3cdad54e04ccb3cc: Status 404 returned error can't find the container with id 0001879dffa1c15fa6ab82f0ba7195ce7631560e3042fc2c3cdad54e04ccb3cc
Sep 16 13:32:55.017165 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:55.017129    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50365d01_e172_4f8f_bd73_3f62082b0cf1.slice/crio-219562241afa2cc73fae880d260ca4610f8732e24a7ff1613feff3c07589a40e WatchSource:0}: Error finding container 219562241afa2cc73fae880d260ca4610f8732e24a7ff1613feff3c07589a40e: Status 404 returned error can't find the container with id 219562241afa2cc73fae880d260ca4610f8732e24a7ff1613feff3c07589a40e
Sep 16 13:32:55.025583 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:32:55.025556    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75292479_8bee_4bc8_8263_e77513e1949b.slice/crio-3990782ae96bd9e1fa87172a2b3ecdfbbd22f4d6c9acbf0e302019d860d812ce WatchSource:0}: Error finding container 3990782ae96bd9e1fa87172a2b3ecdfbbd22f4d6c9acbf0e302019d860d812ce: Status 404 returned error can't find the container with id 3990782ae96bd9e1fa87172a2b3ecdfbbd22f4d6c9acbf0e302019d860d812ce
Sep 16 13:32:55.282120 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.282021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:55.282293 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.282175    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:55.282293 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.282266    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:32:56.282246685 +0000 UTC m=+3.965905074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:55.382923 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.382883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:55.383089 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.383045    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:55.383143 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.383124    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret podName:78390bc8-31f3-4f46-81cb-08992be7587b nodeName:}" failed. No retries permitted until 2026-09-16 13:32:56.38310319 +0000 UTC m=+4.066761569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret") pod "global-pull-secret-syncer-sxq2j" (UID: "78390bc8-31f3-4f46-81cb-08992be7587b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:55.483564 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.483510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:55.483764 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.483716    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:32:55.483830 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.483772    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:32:55.483830 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.483789    2580 projected.go:196] Error preparing data for projected volume kube-api-access-974v4 for pod openshift-network-diagnostics/network-check-target-5655w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:55.483957 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.483857    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4 podName:3215ed35-059b-46bb-b6ba-15c279a68841 nodeName:}" failed. No retries permitted until 2026-09-16 13:32:56.483841984 +0000 UTC m=+4.167500357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-974v4" (UniqueName: "kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4") pod "network-check-target-5655w" (UID: "3215ed35-059b-46bb-b6ba-15c279a68841") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:55.709095 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.708968    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:27:55 +0000 UTC" deadline="2028-05-14 02:11:43.445637162 +0000 UTC"
Sep 16 13:32:55.709095 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.709043    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14532h38m47.736599156s"
Sep 16 13:32:55.797538 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.797500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:55.797736 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.797646    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:32:55.798122 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.798099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:55.798239 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:55.798216    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:32:55.808083 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.807640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"219562241afa2cc73fae880d260ca4610f8732e24a7ff1613feff3c07589a40e"}
Sep 16 13:32:55.813610 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.811425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l6pz6" event={"ID":"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c","Type":"ContainerStarted","Data":"0001879dffa1c15fa6ab82f0ba7195ce7631560e3042fc2c3cdad54e04ccb3cc"}
Sep 16 13:32:55.813610 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.813146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h4jzc" event={"ID":"ed95382d-1fdc-40dd-a557-b20b2e4cb038","Type":"ContainerStarted","Data":"e572fed6e97ce244015c403476f60879117348bc8627bb5373493c0394c455f9"}
Sep 16 13:32:55.816136 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.816057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vj42r" event={"ID":"4280dfa4-c6ca-4824-8d64-e4d90bf2f176","Type":"ContainerStarted","Data":"6023f1dae31ef35f964cbd9a57b11e94fd70d0a8cae84d4c33bedda45cac7f61"}
Sep 16 13:32:55.817749 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.817677    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerStarted","Data":"e2b64c31b9266109207ea444e3c33044a2b8bb984bc69ed84e2180393d08b952"}
Sep 16 13:32:55.819366 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.819310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kf6d5" event={"ID":"b83024eb-d98d-4dc6-8e35-81d015973261","Type":"ContainerStarted","Data":"d8752476cb1543dafd71885dcabae56e1ff922167d3c1705bc7a41c12ac59c62"}
Sep 16 13:32:55.822214 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.822145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k6cv" event={"ID":"75292479-8bee-4bc8-8263-e77513e1949b","Type":"ContainerStarted","Data":"3990782ae96bd9e1fa87172a2b3ecdfbbd22f4d6c9acbf0e302019d860d812ce"}
Sep 16 13:32:55.826744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.826710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vcjzc" event={"ID":"6a28437e-71de-45c2-8064-02e2d89aa324","Type":"ContainerStarted","Data":"72a723317f49893295f2690774868c43af1a410f5ce16efb181cb9b5c87380ef"}
Sep 16 13:32:55.829162 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:55.829106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc" event={"ID":"90074422-7839-4850-af0a-7c2d48280472","Type":"ContainerStarted","Data":"236080513a8b481c2e65cbd2532ce6bf2564667ae0c63499113542fa3f643e00"}
Sep 16 13:32:56.291654 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:56.291612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:56.291838 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.291790    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:56.291919 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.291859    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:32:58.291838895 +0000 UTC m=+5.975497269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:56.392724 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:56.392680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:56.392925 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.392866    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:56.393031 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.392937    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret podName:78390bc8-31f3-4f46-81cb-08992be7587b nodeName:}" failed. No retries permitted until 2026-09-16 13:32:58.392917122 +0000 UTC m=+6.076575510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret") pod "global-pull-secret-syncer-sxq2j" (UID: "78390bc8-31f3-4f46-81cb-08992be7587b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:56.494035 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:56.493994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:56.494268 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.494227    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:32:56.494268 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.494249    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:32:56.494268 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.494262    2580 projected.go:196] Error preparing data for projected volume kube-api-access-974v4 for pod openshift-network-diagnostics/network-check-target-5655w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:56.494467 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.494326    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4 podName:3215ed35-059b-46bb-b6ba-15c279a68841 nodeName:}" failed. No retries permitted until 2026-09-16 13:32:58.494307533 +0000 UTC m=+6.177965915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-974v4" (UniqueName: "kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4") pod "network-check-target-5655w" (UID: "3215ed35-059b-46bb-b6ba-15c279a68841") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:56.709726 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:56.709622    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 13:27:55 +0000 UTC" deadline="2028-05-30 21:51:39.609887392 +0000 UTC"
Sep 16 13:32:56.709726 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:56.709672    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14936h18m42.900221998s"
Sep 16 13:32:56.796608 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:56.796570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:56.796807 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:56.796731    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:32:57.796576 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:57.796440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:57.797050 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:57.796580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:32:57.797050 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:57.796932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:57.797175 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:57.797058    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:32:58.310566 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:58.310519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:58.310772 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.310707    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:58.310839 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.310793    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:02.310771615 +0000 UTC m=+9.994429990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:32:58.411124 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:58.411082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:58.411339 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.411278    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:58.411411 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.411354    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret podName:78390bc8-31f3-4f46-81cb-08992be7587b nodeName:}" failed. No retries permitted until 2026-09-16 13:33:02.411336196 +0000 UTC m=+10.094994589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret") pod "global-pull-secret-syncer-sxq2j" (UID: "78390bc8-31f3-4f46-81cb-08992be7587b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:32:58.512250 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:58.512196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:58.512440 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.512421    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:32:58.512501 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.512445    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:32:58.512501 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.512458    2580 projected.go:196] Error preparing data for projected volume kube-api-access-974v4 for pod openshift-network-diagnostics/network-check-target-5655w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:58.512601 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.512522    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4 podName:3215ed35-059b-46bb-b6ba-15c279a68841 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:02.512503059 +0000 UTC m=+10.196161434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-974v4" (UniqueName: "kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4") pod "network-check-target-5655w" (UID: "3215ed35-059b-46bb-b6ba-15c279a68841") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:32:58.797320 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:58.797225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:32:58.797781 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:58.797395    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:32:59.796730 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:59.796689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:32:59.796941 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:59.796823    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:32:59.796941 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:32:59.796906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:32:59.797073 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:32:59.797000    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:00.797245 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:00.796693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:00.797245 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:00.796849    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:01.796811 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:01.796775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:01.797030 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:01.796891    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:01.797285 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:01.797267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:01.797642 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:01.797360    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:02.345745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:02.345705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:02.345990 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.345878    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:33:02.345990 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.345964    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:10.345928744 +0000 UTC m=+18.029587124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:33:02.447927 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:02.447175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:02.447927 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.447332    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:33:02.447927 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.447403    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret podName:78390bc8-31f3-4f46-81cb-08992be7587b nodeName:}" failed. No retries permitted until 2026-09-16 13:33:10.447382578 +0000 UTC m=+18.131040958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret") pod "global-pull-secret-syncer-sxq2j" (UID: "78390bc8-31f3-4f46-81cb-08992be7587b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:33:02.547901 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:02.547860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:02.548123 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.548070    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:33:02.548123 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.548094    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:33:02.548123 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.548108    2580 projected.go:196] Error preparing data for projected volume kube-api-access-974v4 for pod openshift-network-diagnostics/network-check-target-5655w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:33:02.548292 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.548179    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4 podName:3215ed35-059b-46bb-b6ba-15c279a68841 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:10.548160257 +0000 UTC m=+18.231818637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-974v4" (UniqueName: "kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4") pod "network-check-target-5655w" (UID: "3215ed35-059b-46bb-b6ba-15c279a68841") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:33:02.798341 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:02.798239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:02.798823 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:02.798393    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:03.797160 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:03.797123    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:03.797350 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:03.797259    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:03.797350 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:03.797298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:03.797485 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:03.797371    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:04.797475 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:04.797391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:04.798085 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:04.797548    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:05.797121 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:05.796603    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:05.797121 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:05.796645    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:05.797121 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:05.796745    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:05.797121 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:05.796820    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:06.797374 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:06.797337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:06.797771 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:06.797468    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:07.797101 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:07.797063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:07.797301 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:07.797063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:07.797301 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:07.797218    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:07.797301 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:07.797288    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:08.797468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:08.797435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:08.797880 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:08.797573    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:09.796514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:09.796483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:09.796703 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:09.796481    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:09.796703 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:09.796604    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:09.796810 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:09.796724    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:10.410579 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:10.410539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:10.410985 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.410698    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:33:10.410985 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.410780    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:26.410759917 +0000 UTC m=+34.094418296 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:33:10.511061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:10.511029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:10.511244 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.511188    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:33:10.511308 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.511258    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret podName:78390bc8-31f3-4f46-81cb-08992be7587b nodeName:}" failed. No retries permitted until 2026-09-16 13:33:26.511240948 +0000 UTC m=+34.194899321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret") pod "global-pull-secret-syncer-sxq2j" (UID: "78390bc8-31f3-4f46-81cb-08992be7587b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:33:10.612297 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:10.612260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:10.612471 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.612448    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:33:10.612551 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.612480    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:33:10.612551 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.612494    2580 projected.go:196] Error preparing data for projected volume kube-api-access-974v4 for pod openshift-network-diagnostics/network-check-target-5655w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:33:10.612644 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.612555    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4 podName:3215ed35-059b-46bb-b6ba-15c279a68841 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:26.612538386 +0000 UTC m=+34.296196759 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-974v4" (UniqueName: "kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4") pod "network-check-target-5655w" (UID: "3215ed35-059b-46bb-b6ba-15c279a68841") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:33:10.796557 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:10.796478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:10.796703 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:10.796601    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:11.796802 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:11.796763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:11.797388 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:11.796763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:11.797388 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:11.796887    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:11.797388 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:11.796977    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:12.797221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:12.797183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:12.797652 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:12.797294    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:13.796698 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.796463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:13.796698 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.796548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:13.796698 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:13.796789    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:13.796698 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:13.796839    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:13.884787 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.884748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal" event={"ID":"4054c0090de8f54aed802a5d555be580","Type":"ContainerStarted","Data":"df7240381c3c140cdfa22848ba33a6c2bb4659d157f5a61aaf388171e1c47611"}
Sep 16 13:33:13.893416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.893352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k6cv" event={"ID":"75292479-8bee-4bc8-8263-e77513e1949b","Type":"ContainerStarted","Data":"098e68e6ec44ecb6d356549ca24b18cd090be9635253c090a8f44e4d55377090"}
Sep 16 13:33:13.904731 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.904653    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:33:13.905861 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.905801    2580 generic.go:356] "Generic (PLEG): container finished" podID="50365d01-e172-4f8f-bd73-3f62082b0cf1" containerID="74326001f419970cc5f01ca1a4f5e7838dab0ade4dcce37ab32d657e25d61563" exitCode=1
Sep 16 13:33:13.905991 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.905892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerDied","Data":"74326001f419970cc5f01ca1a4f5e7838dab0ade4dcce37ab32d657e25d61563"}
Sep 16 13:33:13.906056 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.905996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"1d4d9277b89232449a361b240e459c38866a9f5c2a4e5aac32e3166a922293ba"}
Sep 16 13:33:13.907453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.906776    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-207.ec2.internal" podStartSLOduration=19.906760906 podStartE2EDuration="19.906760906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:13.905938174 +0000 UTC m=+21.589596569" watchObservedRunningTime="2026-09-16 13:33:13.906760906 +0000 UTC m=+21.590419300"
Sep 16 13:33:13.913840 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.913789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l6pz6" event={"ID":"75ba721d-4a94-4ae0-89b6-a31d4d62ee3c","Type":"ContainerStarted","Data":"97365a9441f574cc502f63423168a2ea1510cb0c245bd87e022393862c831fc4"}
Sep 16 13:33:13.938513 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.938470    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4k6cv" podStartSLOduration=1.260204869 podStartE2EDuration="19.938450798s" totalImagesPullingTime="18.678245929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:13.922836344 +0000 UTC m=+21.606494739" watchObservedRunningTime="2026-09-16 13:33:13.938450798 +0000 UTC m=+21.622109189"
Sep 16 13:33:13.938722 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:13.938699    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l6pz6" podStartSLOduration=1.639927323 podStartE2EDuration="19.938692216s" totalImagesPullingTime="18.298764893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:13.938313374 +0000 UTC m=+21.621971768" watchObservedRunningTime="2026-09-16 13:33:13.938692216 +0000 UTC m=+21.622350613"
Sep 16 13:33:14.796958 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.796769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:14.797186 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:14.797076    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:14.917862 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.917830    2580 generic.go:356] "Generic (PLEG): container finished" podID="a6be15198fb113e0b9a0291128436e59" containerID="4ede27eaa5c113b9454aa13ce17a58a4a2fb75621d2de10d119c194d87eae948" exitCode=0
Sep 16 13:33:14.918622 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.917924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" event={"ID":"a6be15198fb113e0b9a0291128436e59","Type":"ContainerDied","Data":"4ede27eaa5c113b9454aa13ce17a58a4a2fb75621d2de10d119c194d87eae948"}
Sep 16 13:33:14.921385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.921363    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:33:14.921791 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.921764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"4809c1ecc1cb3e131d651413c77a10560671a0b1e1316f0236cf2f645bb29923"}
Sep 16 13:33:14.921921 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.921844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"8528eed35009af6863f9eb2916aa04863e74ed0282d84c360df0484ab494b35d"}
Sep 16 13:33:14.921921 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.921878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"b55b6087ac23a00b10b11d2a9f10a49cfc0f74b632af89aa5883e83e96d5663a"}
Sep 16 13:33:14.922039 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.921933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"b66518f67744f0c4294e830efc69cc6e0271f79ada36414ca4382ae6a2067fde"}
Sep 16 13:33:14.923328 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.923304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h4jzc" event={"ID":"ed95382d-1fdc-40dd-a557-b20b2e4cb038","Type":"ContainerStarted","Data":"2428b555a7f61e0263cedff8f7b9bef30d481a0689b99e687b709500eb721fb3"}
Sep 16 13:33:14.924868 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.924840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vj42r" event={"ID":"4280dfa4-c6ca-4824-8d64-e4d90bf2f176","Type":"ContainerStarted","Data":"65144a3debd9e6ee0889c222afbdb1b27a23d5238b31256b09327cab57a1e7df"}
Sep 16 13:33:14.926376 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.926353    2580 generic.go:356] "Generic (PLEG): container finished" podID="46a9e2e1-e39a-490d-a9ad-e919b33157dd" containerID="f52ec90599e2d2367b25b84c3745c798d4fe3977efe07a5d0260a388ac188f47" exitCode=0
Sep 16 13:33:14.926460 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.926414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerDied","Data":"f52ec90599e2d2367b25b84c3745c798d4fe3977efe07a5d0260a388ac188f47"}
Sep 16 13:33:14.927949 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.927924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vcjzc" event={"ID":"6a28437e-71de-45c2-8064-02e2d89aa324","Type":"ContainerStarted","Data":"2c9e7dc8e826acba8826601b4ea7eb27c89fa674c75bbf6a4f9484c0c25203ff"}
Sep 16 13:33:14.929533 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.929504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc" event={"ID":"90074422-7839-4850-af0a-7c2d48280472","Type":"ContainerStarted","Data":"db18f5c4e3e073141158144999e8d6737e7fa3ce66e2c941f2ee51697c76d105"}
Sep 16 13:33:14.961454 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.961396    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vj42r" podStartSLOduration=3.42546221 podStartE2EDuration="21.961379315s" totalImagesPullingTime="18.535917105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:14.961285519 +0000 UTC m=+22.644943913" watchObservedRunningTime="2026-09-16 13:33:14.961379315 +0000 UTC m=+22.645037710"
Sep 16 13:33:14.973658 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.973603    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h4jzc" podStartSLOduration=3.670487799 podStartE2EDuration="21.973584915s" totalImagesPullingTime="18.303097116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:14.973140939 +0000 UTC m=+22.656799338" watchObservedRunningTime="2026-09-16 13:33:14.973584915 +0000 UTC m=+22.657243309"
Sep 16 13:33:14.985316 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:14.985253    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vcjzc" podStartSLOduration=3.45660507 podStartE2EDuration="21.985233432s" totalImagesPullingTime="18.528628362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:14.984797698 +0000 UTC m=+22.668456094" watchObservedRunningTime="2026-09-16 13:33:14.985233432 +0000 UTC m=+22.668891826"
Sep 16 13:33:15.202571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.202540    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 13:33:15.732943 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.732826    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T13:33:15.202557096Z","UUID":"80648ce5-5839-4856-85aa-4422fe84eb18","Handler":null,"Name":"","Endpoint":""}
Sep 16 13:33:15.735772 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.735748    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 13:33:15.735917 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.735781    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 13:33:15.796927 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.796871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:15.797069 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.796873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:15.797069 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:15.796979    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:15.797069 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:15.797050    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:15.934057 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.934023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" event={"ID":"a6be15198fb113e0b9a0291128436e59","Type":"ContainerStarted","Data":"c3e57ae2b09a6512609285e720e6cdf87869c2750d72b5c44cedd5f3353f11b9"}
Sep 16 13:33:15.935851 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.935819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kf6d5" event={"ID":"b83024eb-d98d-4dc6-8e35-81d015973261","Type":"ContainerStarted","Data":"81fd5636cae34677852485d2a21bbffc798108e77a60e64ca7ab890ff48648eb"}
Sep 16 13:33:15.937976 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.937947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc" event={"ID":"90074422-7839-4850-af0a-7c2d48280472","Type":"ContainerStarted","Data":"e0201d245104959d5881e9ef9f6e2c10ac9f7ab4ca145e4fe0646a9692de30be"}
Sep 16 13:33:15.951103 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.951064    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-207.ec2.internal" podStartSLOduration=21.951052766 podStartE2EDuration="21.951052766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:15.950939451 +0000 UTC m=+23.634597845" watchObservedRunningTime="2026-09-16 13:33:15.951052766 +0000 UTC m=+23.634711160"
Sep 16 13:33:15.963674 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:15.963625    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kf6d5" podStartSLOduration=11.692940196 podStartE2EDuration="22.963613676s" totalImagesPullingTime="11.27067348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:15.963142353 +0000 UTC m=+23.646800747" watchObservedRunningTime="2026-09-16 13:33:15.963613676 +0000 UTC m=+23.647272084"
Sep 16 13:33:16.799647 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:16.799484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:16.799807 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:16.799738    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:16.943719 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:16.943680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc" event={"ID":"90074422-7839-4850-af0a-7c2d48280472","Type":"ContainerStarted","Data":"5a1008cd8c65f0628c68e23ef9dd59d427f2edbb5df705693f54bbbb3461b775"}
Sep 16 13:33:16.947546 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:16.947520    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:33:16.948013 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:16.947990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"2cc862a8ae7d3971db697108abfe3c8d68a75a159e391e1c0d686e0b043eb8c9"}
Sep 16 13:33:16.959380 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:16.959342    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pchtc" podStartSLOduration=6.921941177 podStartE2EDuration="23.959329199s" totalImagesPullingTime="17.037388022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:16.959263677 +0000 UTC m=+24.642922082" watchObservedRunningTime="2026-09-16 13:33:16.959329199 +0000 UTC m=+24.642987592"
Sep 16 13:33:17.797114 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:17.797078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:17.797315 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:17.797197    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:17.797315 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:17.797247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:17.797438 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:17.797355    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:18.532371 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:18.532343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:33:18.533536 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:18.533290    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:33:18.797284 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:18.797211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:18.797513 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:18.797409    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:18.956877 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:18.956850    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:33:18.957397 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:18.957361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"3f26956e3feba499374f34569254681ac7499fa810def158493ac4f5345945bd"}
Sep 16 13:33:18.957765 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:18.957750    2580 scope.go:117] "RemoveContainer" containerID="74326001f419970cc5f01ca1a4f5e7838dab0ade4dcce37ab32d657e25d61563"
Sep 16 13:33:19.797197 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:19.797032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:19.797628 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:19.797084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:19.797628 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:19.797311    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:19.797628 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:19.797376    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:20.357347 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.357320    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sxq2j"]
Sep 16 13:33:20.357497 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.357481    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:20.357629 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:20.357602    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:20.360608 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.360516    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5655w"]
Sep 16 13:33:20.360672 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.360642    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:20.360751 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:20.360720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:20.361353 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.361328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wszjx"]
Sep 16 13:33:20.361458 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.361442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:20.361617 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:20.361592    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:20.963981 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.963945    2580 generic.go:356] "Generic (PLEG): container finished" podID="46a9e2e1-e39a-490d-a9ad-e919b33157dd" containerID="984cdc71eeea3ab414adfde90369b268e02b20b0cebc92f4c5f47705111d1eb5" exitCode=0
Sep 16 13:33:20.964745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.964037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerDied","Data":"984cdc71eeea3ab414adfde90369b268e02b20b0cebc92f4c5f47705111d1eb5"}
Sep 16 13:33:20.968202 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.968185    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:33:20.968519 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.968500    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" event={"ID":"50365d01-e172-4f8f-bd73-3f62082b0cf1","Type":"ContainerStarted","Data":"461252ce6b8354ccb3a20546fcb76c371cc7c5f028e9b891d80d24b70f2e2369"}
Sep 16 13:33:20.968827 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.968804    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:33:20.968937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.968874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:33:20.968937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.968936    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:33:20.983780 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.983757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:33:20.983981 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:20.983964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:33:21.002883 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:21.002843    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56" podStartSLOduration=8.45452085 podStartE2EDuration="27.002829937s" totalImagesPullingTime="18.548309087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:21.002398454 +0000 UTC m=+28.686056847" watchObservedRunningTime="2026-09-16 13:33:21.002829937 +0000 UTC m=+28.686488323"
Sep 16 13:33:21.797365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:21.797328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:21.797365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:21.797363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:21.797568 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:21.797445    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:21.797568 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:21.797499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:21.797644 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:21.797596    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:21.797691 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:21.797672    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:22.975256 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:22.975066    2580 generic.go:356] "Generic (PLEG): container finished" podID="46a9e2e1-e39a-490d-a9ad-e919b33157dd" containerID="069d182a738e50de956e0799009453cd081989777ce02a6fa8736f8d87be21e4" exitCode=0
Sep 16 13:33:22.975659 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:22.975147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerDied","Data":"069d182a738e50de956e0799009453cd081989777ce02a6fa8736f8d87be21e4"}
Sep 16 13:33:23.797077 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:23.797046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:23.797077 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:23.797062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:23.797272 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:23.797141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:23.797272 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:23.797146    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:23.797272 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:23.797227    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:23.797365 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:23.797296    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:24.079659 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:24.079584    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:33:24.080018 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:24.079689    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 13:33:24.080154 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:24.080139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vj42r"
Sep 16 13:33:24.982181 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:24.982150    2580 generic.go:356] "Generic (PLEG): container finished" podID="46a9e2e1-e39a-490d-a9ad-e919b33157dd" containerID="aa106fe70d578bf56b079a02ecca00e60acebd3e7d1fe138dae43350a11cf5f7" exitCode=0
Sep 16 13:33:24.982342 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:24.982240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerDied","Data":"aa106fe70d578bf56b079a02ecca00e60acebd3e7d1fe138dae43350a11cf5f7"}
Sep 16 13:33:25.796454 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:25.796400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:25.796454 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:25.796448    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:25.796924 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:25.796400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:25.796924 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:25.796531    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wszjx" podUID="6ea9a2a4-11ee-4027-affe-da9400ccd248"
Sep 16 13:33:25.796924 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:25.796602    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sxq2j" podUID="78390bc8-31f3-4f46-81cb-08992be7587b"
Sep 16 13:33:25.796924 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:25.796670    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5655w" podUID="3215ed35-059b-46bb-b6ba-15c279a68841"
Sep 16 13:33:26.426584 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:26.426552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:26.426829 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.426696    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:33:26.426829 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.426768    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:58.42674932 +0000 UTC m=+66.110407707 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 13:33:26.527257 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:26.527218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:26.527419 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.527369    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 13:33:26.527460 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.527440    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret podName:78390bc8-31f3-4f46-81cb-08992be7587b nodeName:}" failed. No retries permitted until 2026-09-16 13:33:58.527422062 +0000 UTC m=+66.211080450 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret") pod "global-pull-secret-syncer-sxq2j" (UID: "78390bc8-31f3-4f46-81cb-08992be7587b") : object "kube-system"/"original-pull-secret" not registered
Sep 16 13:33:26.627760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:26.627727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:26.627963 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.627924    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 13:33:26.627963 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.627950    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 13:33:26.627963 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.627961    2580 projected.go:196] Error preparing data for projected volume kube-api-access-974v4 for pod openshift-network-diagnostics/network-check-target-5655w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:33:26.628144 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:26.628021    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4 podName:3215ed35-059b-46bb-b6ba-15c279a68841 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:58.628001879 +0000 UTC m=+66.311660258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-974v4" (UniqueName: "kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4") pod "network-check-target-5655w" (UID: "3215ed35-059b-46bb-b6ba-15c279a68841") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 13:33:27.570152 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.570067    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-207.ec2.internal" event="NodeReady"
Sep 16 13:33:27.570548 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.570225    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 13:33:27.602103 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.602074    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"]
Sep 16 13:33:27.603996 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.603973    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.605447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.605424    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"]
Sep 16 13:33:27.606243 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.606220    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6scpb\""
Sep 16 13:33:27.606243 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.606222    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 13:33:27.606677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.606619    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 13:33:27.606677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.606617    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 13:33:27.606677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.606650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 16 13:33:27.610920 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.610886    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"]
Sep 16 13:33:27.611026 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.611004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.612726 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.612668    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74fd95df67-ttg94"]
Sep 16 13:33:27.612823 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.612795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.613524 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.613333    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 16 13:33:27.613524 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.613410    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 16 13:33:27.613524 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.613457    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 16 13:33:27.614277 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.614136    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 16 13:33:27.614951 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.614676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.614951 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.614806    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 16 13:33:27.617537 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.616730    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 13:33:27.617537 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.616797    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c5bs9\""
Sep 16 13:33:27.617537 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.617007    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 13:33:27.617537 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.617153    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 13:33:27.617537 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.617195    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"]
Sep 16 13:33:27.617882 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.617852    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-snmh9"]
Sep 16 13:33:27.622677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.621954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"]
Sep 16 13:33:27.622677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.622054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:27.622885 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.622776    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"]
Sep 16 13:33:27.622885 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.622871    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 13:33:27.625767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.624681    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 13:33:27.625767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.624703    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 13:33:27.625767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.624883    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xft82"]
Sep 16 13:33:27.625767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.624952    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2fr7x\""
Sep 16 13:33:27.625767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.625263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 13:33:27.626943 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.626924    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74fd95df67-ttg94"]
Sep 16 13:33:27.627047 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.626956    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-snmh9"]
Sep 16 13:33:27.627047 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.627002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.629303 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.629286    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 13:33:27.629488 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.629473    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 13:33:27.629601 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.629583    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nnkqw\""
Sep 16 13:33:27.637220 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.637197    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xft82"]
Sep 16 13:33:27.735930 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.735884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-ca\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.736088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.735935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b59d0af7-1225-42f1-9a74-29a6ad69b8df-ca-trust-extracted\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.736088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.735963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-certificates\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.736088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.735983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5b7f\" (UniqueName: \"kubernetes.io/projected/9fa7eea5-6a00-4cc2-9860-359ca3284c14-kube-api-access-j5b7f\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.736088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-hub\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.736088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-bound-sa-token\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.736244 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d56af732-9bed-4dd8-9442-1d0d6ff94734-config-volume\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.736244 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzv6m\" (UniqueName: \"kubernetes.io/projected/4f237f6d-0a4f-4559-b2fb-256e9598a755-kube-api-access-wzv6m\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.736244 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9fa7eea5-6a00-4cc2-9860-359ca3284c14-klusterlet-config\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.736244 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.736244 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mr8v\" (UniqueName: \"kubernetes.io/projected/d56af732-9bed-4dd8-9442-1d0d6ff94734-kube-api-access-5mr8v\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.736385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f237f6d-0a4f-4559-b2fb-256e9598a755-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.736385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nszw7\" (UniqueName: \"kubernetes.io/projected/493adda8-df96-418e-8f5d-b0b611d13187-kube-api-access-nszw7\") pod \"ingress-canary-snmh9\" (UID: \"493adda8-df96-418e-8f5d-b0b611d13187\") " pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:27.736385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-image-registry-private-configuration\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.736385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fa7eea5-6a00-4cc2-9860-359ca3284c14-tmp\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.736385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56af732-9bed-4dd8-9442-1d0d6ff94734-tmp-dir\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.736385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.736554 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-trusted-ca\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.736554 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5769d743-ff5f-4061-adf6-d555412d32b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bfc9c97b-7wqzc\" (UID: \"5769d743-ff5f-4061-adf6-d555412d32b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.736554 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-installation-pull-secrets\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.736554 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/493adda8-df96-418e-8f5d-b0b611d13187-cert\") pod \"ingress-canary-snmh9\" (UID: \"493adda8-df96-418e-8f5d-b0b611d13187\") " pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:27.736554 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7cps\" (UniqueName: \"kubernetes.io/projected/5769d743-ff5f-4061-adf6-d555412d32b1-kube-api-access-k7cps\") pod \"managed-serviceaccount-addon-agent-bfc9c97b-7wqzc\" (UID: \"5769d743-ff5f-4061-adf6-d555412d32b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.736554 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.736723 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.736723 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.736617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2stf\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-kube-api-access-m2stf\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.797337 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.797302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:27.797337 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.797331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:27.797337 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.797339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:27.799757 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.799729    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 13:33:27.799919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.799824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-69tl6\""
Sep 16 13:33:27.799919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.799824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2cz7s\""
Sep 16 13:33:27.799919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.799891    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 13:33:27.800114 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.799824    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 13:33:27.800114 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.799978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 13:33:27.837558 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-hub\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.837684 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-bound-sa-token\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.837684 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d56af732-9bed-4dd8-9442-1d0d6ff94734-config-volume\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.837684 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzv6m\" (UniqueName: \"kubernetes.io/projected/4f237f6d-0a4f-4559-b2fb-256e9598a755-kube-api-access-wzv6m\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.837684 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9fa7eea5-6a00-4cc2-9860-359ca3284c14-klusterlet-config\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.837684 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mr8v\" (UniqueName: \"kubernetes.io/projected/d56af732-9bed-4dd8-9442-1d0d6ff94734-kube-api-access-5mr8v\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f237f6d-0a4f-4559-b2fb-256e9598a755-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nszw7\" (UniqueName: \"kubernetes.io/projected/493adda8-df96-418e-8f5d-b0b611d13187-kube-api-access-nszw7\") pod \"ingress-canary-snmh9\" (UID: \"493adda8-df96-418e-8f5d-b0b611d13187\") " pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-image-registry-private-configuration\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fa7eea5-6a00-4cc2-9860-359ca3284c14-tmp\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56af732-9bed-4dd8-9442-1d0d6ff94734-tmp-dir\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-trusted-ca\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5769d743-ff5f-4061-adf6-d555412d32b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bfc9c97b-7wqzc\" (UID: \"5769d743-ff5f-4061-adf6-d555412d32b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-installation-pull-secrets\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.837937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/493adda8-df96-418e-8f5d-b0b611d13187-cert\") pod \"ingress-canary-snmh9\" (UID: \"493adda8-df96-418e-8f5d-b0b611d13187\") " pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7cps\" (UniqueName: \"kubernetes.io/projected/5769d743-ff5f-4061-adf6-d555412d32b1-kube-api-access-k7cps\") pod \"managed-serviceaccount-addon-agent-bfc9c97b-7wqzc\" (UID: \"5769d743-ff5f-4061-adf6-d555412d32b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.837989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2stf\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-kube-api-access-m2stf\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-ca\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b59d0af7-1225-42f1-9a74-29a6ad69b8df-ca-trust-extracted\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-certificates\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5b7f\" (UniqueName: \"kubernetes.io/projected/9fa7eea5-6a00-4cc2-9860-359ca3284c14-kube-api-access-j5b7f\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838212    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d56af732-9bed-4dd8-9442-1d0d6ff94734-config-volume\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.838396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fa7eea5-6a00-4cc2-9860-359ca3284c14-tmp\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.838855 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.838466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d56af732-9bed-4dd8-9442-1d0d6ff94734-tmp-dir\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.839080 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.839049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b59d0af7-1225-42f1-9a74-29a6ad69b8df-ca-trust-extracted\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.839895 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.839843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-trusted-ca\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.840016 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:27.839962    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:33:27.840016 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:27.839980    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74fd95df67-ttg94: secret "image-registry-tls" not found
Sep 16 13:33:27.840112 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.840011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-certificates\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.840112 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:27.840045    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls podName:b59d0af7-1225-42f1-9a74-29a6ad69b8df nodeName:}" failed. No retries permitted until 2026-09-16 13:33:28.340026155 +0000 UTC m=+36.023684542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls") pod "image-registry-74fd95df67-ttg94" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df") : secret "image-registry-tls" not found
Sep 16 13:33:27.840876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.840850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f237f6d-0a4f-4559-b2fb-256e9598a755-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.840992 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:27.840973    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:33:27.841066 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:27.841029    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls podName:d56af732-9bed-4dd8-9442-1d0d6ff94734 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:28.341013003 +0000 UTC m=+36.024671390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls") pod "dns-default-xft82" (UID: "d56af732-9bed-4dd8-9442-1d0d6ff94734") : secret "dns-default-metrics-tls" not found
Sep 16 13:33:27.843585 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.843107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/493adda8-df96-418e-8f5d-b0b611d13187-cert\") pod \"ingress-canary-snmh9\" (UID: \"493adda8-df96-418e-8f5d-b0b611d13187\") " pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:27.843585 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.843544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9fa7eea5-6a00-4cc2-9860-359ca3284c14-klusterlet-config\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.844672 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.844293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5769d743-ff5f-4061-adf6-d555412d32b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bfc9c97b-7wqzc\" (UID: \"5769d743-ff5f-4061-adf6-d555412d32b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.844672 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.844395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-image-registry-private-configuration\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.845315 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.845288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.845749 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.845724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-bound-sa-token\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.846577 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.846035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.846577 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.846503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-ca\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.846577 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.846538    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4f237f6d-0a4f-4559-b2fb-256e9598a755-hub\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.846790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.846762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-installation-pull-secrets\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.847547 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.847520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7cps\" (UniqueName: \"kubernetes.io/projected/5769d743-ff5f-4061-adf6-d555412d32b1-kube-api-access-k7cps\") pod \"managed-serviceaccount-addon-agent-bfc9c97b-7wqzc\" (UID: \"5769d743-ff5f-4061-adf6-d555412d32b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.848030 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.847894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5b7f\" (UniqueName: \"kubernetes.io/projected/9fa7eea5-6a00-4cc2-9860-359ca3284c14-kube-api-access-j5b7f\") pod \"klusterlet-addon-workmgr-54d887877d-h6brg\" (UID: \"9fa7eea5-6a00-4cc2-9860-359ca3284c14\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.848273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.848215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzv6m\" (UniqueName: \"kubernetes.io/projected/4f237f6d-0a4f-4559-b2fb-256e9598a755-kube-api-access-wzv6m\") pod \"cluster-proxy-proxy-agent-6f74495c67-55l49\" (UID: \"4f237f6d-0a4f-4559-b2fb-256e9598a755\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.848976 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.848955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2stf\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-kube-api-access-m2stf\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:27.849276 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.849255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mr8v\" (UniqueName: \"kubernetes.io/projected/d56af732-9bed-4dd8-9442-1d0d6ff94734-kube-api-access-5mr8v\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:27.849814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.849798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nszw7\" (UniqueName: \"kubernetes.io/projected/493adda8-df96-418e-8f5d-b0b611d13187-kube-api-access-nszw7\") pod \"ingress-canary-snmh9\" (UID: \"493adda8-df96-418e-8f5d-b0b611d13187\") " pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:27.923707 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.923646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"
Sep 16 13:33:27.932035 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.931971    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:33:27.949026 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.949000    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:27.954605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.954568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vj42r_4280dfa4-c6ca-4824-8d64-e4d90bf2f176/konnectivity-agent/0.log" limit=1024
Sep 16 13:33:27.962468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:27.962265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-snmh9"
Sep 16 13:33:28.032476 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.031840    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vj42r_4280dfa4-c6ca-4824-8d64-e4d90bf2f176/konnectivity-agent/0.log" limit=1024
Sep 16 13:33:28.222385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.222358    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc"]
Sep 16 13:33:28.225826 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.225774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"]
Sep 16 13:33:28.226817 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:33:28.226791    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5769d743_ff5f_4061_adf6_d555412d32b1.slice/crio-e3cfc1286d1bb2742cd11850595fb435bd13cacad2a2bcddc925b5dc8b464b03 WatchSource:0}: Error finding container e3cfc1286d1bb2742cd11850595fb435bd13cacad2a2bcddc925b5dc8b464b03: Status 404 returned error can't find the container with id e3cfc1286d1bb2742cd11850595fb435bd13cacad2a2bcddc925b5dc8b464b03
Sep 16 13:33:28.228842 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:33:28.228821    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f237f6d_0a4f_4559_b2fb_256e9598a755.slice/crio-94e37957ddffb0dc6e1f066be0a2176321d14908c88d3acc4eff71ee496b136b WatchSource:0}: Error finding container 94e37957ddffb0dc6e1f066be0a2176321d14908c88d3acc4eff71ee496b136b: Status 404 returned error can't find the container with id 94e37957ddffb0dc6e1f066be0a2176321d14908c88d3acc4eff71ee496b136b
Sep 16 13:33:28.238473 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.238440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"]
Sep 16 13:33:28.244636 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:33:28.244611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa7eea5_6a00_4cc2_9860_359ca3284c14.slice/crio-1ea6a2a95a12329eb8f5699ce3adfe8afb7b18eef11b7648aebf0dee85be2ce0 WatchSource:0}: Error finding container 1ea6a2a95a12329eb8f5699ce3adfe8afb7b18eef11b7648aebf0dee85be2ce0: Status 404 returned error can't find the container with id 1ea6a2a95a12329eb8f5699ce3adfe8afb7b18eef11b7648aebf0dee85be2ce0
Sep 16 13:33:28.258688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.258662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-snmh9"]
Sep 16 13:33:28.261893 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:33:28.261868    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493adda8_df96_418e_8f5d_b0b611d13187.slice/crio-bbc5abedf596ba75388484816a4cb784a6a921fafcc4eb7ea057ca550514ad95 WatchSource:0}: Error finding container bbc5abedf596ba75388484816a4cb784a6a921fafcc4eb7ea057ca550514ad95: Status 404 returned error can't find the container with id bbc5abedf596ba75388484816a4cb784a6a921fafcc4eb7ea057ca550514ad95
Sep 16 13:33:28.344141 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.344107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:28.344319 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.344175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:28.344319 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:28.344276    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:33:28.344395 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:28.344329    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:33:28.344395 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:28.344349    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74fd95df67-ttg94: secret "image-registry-tls" not found
Sep 16 13:33:28.344395 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:28.344357    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls podName:d56af732-9bed-4dd8-9442-1d0d6ff94734 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:29.344337694 +0000 UTC m=+37.027996080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls") pod "dns-default-xft82" (UID: "d56af732-9bed-4dd8-9442-1d0d6ff94734") : secret "dns-default-metrics-tls" not found
Sep 16 13:33:28.344501 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:28.344400    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls podName:b59d0af7-1225-42f1-9a74-29a6ad69b8df nodeName:}" failed. No retries permitted until 2026-09-16 13:33:29.344385648 +0000 UTC m=+37.028044042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls") pod "image-registry-74fd95df67-ttg94" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df") : secret "image-registry-tls" not found
Sep 16 13:33:28.995652 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.995608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" event={"ID":"4f237f6d-0a4f-4559-b2fb-256e9598a755","Type":"ContainerStarted","Data":"94e37957ddffb0dc6e1f066be0a2176321d14908c88d3acc4eff71ee496b136b"}
Sep 16 13:33:28.997144 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.997105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc" event={"ID":"5769d743-ff5f-4061-adf6-d555412d32b1","Type":"ContainerStarted","Data":"e3cfc1286d1bb2742cd11850595fb435bd13cacad2a2bcddc925b5dc8b464b03"}
Sep 16 13:33:28.998683 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:28.998657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-snmh9" event={"ID":"493adda8-df96-418e-8f5d-b0b611d13187","Type":"ContainerStarted","Data":"bbc5abedf596ba75388484816a4cb784a6a921fafcc4eb7ea057ca550514ad95"}
Sep 16 13:33:29.000371 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:29.000344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg" event={"ID":"9fa7eea5-6a00-4cc2-9860-359ca3284c14","Type":"ContainerStarted","Data":"1ea6a2a95a12329eb8f5699ce3adfe8afb7b18eef11b7648aebf0dee85be2ce0"}
Sep 16 13:33:29.354960 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:29.354031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:29.354960 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:29.354132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:29.354960 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:29.354316    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:33:29.354960 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:29.354333    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74fd95df67-ttg94: secret "image-registry-tls" not found
Sep 16 13:33:29.354960 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:29.354392    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls podName:b59d0af7-1225-42f1-9a74-29a6ad69b8df nodeName:}" failed. No retries permitted until 2026-09-16 13:33:31.354374945 +0000 UTC m=+39.038033333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls") pod "image-registry-74fd95df67-ttg94" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df") : secret "image-registry-tls" not found
Sep 16 13:33:29.354960 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:29.354812    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:33:29.354960 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:29.354864    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls podName:d56af732-9bed-4dd8-9442-1d0d6ff94734 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:31.354848049 +0000 UTC m=+39.038506427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls") pod "dns-default-xft82" (UID: "d56af732-9bed-4dd8-9442-1d0d6ff94734") : secret "dns-default-metrics-tls" not found
Sep 16 13:33:31.373403 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:31.372019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:31.373403 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:31.372115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:31.373403 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:31.372229    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:33:31.373403 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:31.372242    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:33:31.373403 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:31.372254    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74fd95df67-ttg94: secret "image-registry-tls" not found
Sep 16 13:33:31.373403 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:31.372304    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls podName:d56af732-9bed-4dd8-9442-1d0d6ff94734 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:35.372286366 +0000 UTC m=+43.055944757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls") pod "dns-default-xft82" (UID: "d56af732-9bed-4dd8-9442-1d0d6ff94734") : secret "dns-default-metrics-tls" not found
Sep 16 13:33:31.373403 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:31.372322    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls podName:b59d0af7-1225-42f1-9a74-29a6ad69b8df nodeName:}" failed. No retries permitted until 2026-09-16 13:33:35.372313223 +0000 UTC m=+43.055971594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls") pod "image-registry-74fd95df67-ttg94" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df") : secret "image-registry-tls" not found
Sep 16 13:33:31.699182 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:31.699149    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vj42r_4280dfa4-c6ca-4824-8d64-e4d90bf2f176/konnectivity-agent/0.log" limit=1024
Sep 16 13:33:31.800478 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:31.800443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vj42r_4280dfa4-c6ca-4824-8d64-e4d90bf2f176/konnectivity-agent/0.log" limit=1024
Sep 16 13:33:31.905061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:31.905025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vj42r_4280dfa4-c6ca-4824-8d64-e4d90bf2f176/konnectivity-agent/0.log" limit=1024
Sep 16 13:33:35.404353 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:35.404311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:35.404930 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:35.404409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:35.404930 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:35.404490    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:33:35.404930 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:35.404518    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74fd95df67-ttg94: secret "image-registry-tls" not found
Sep 16 13:33:35.404930 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:35.404529    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:33:35.404930 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:35.404590    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls podName:d56af732-9bed-4dd8-9442-1d0d6ff94734 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:43.404570149 +0000 UTC m=+51.088228527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls") pod "dns-default-xft82" (UID: "d56af732-9bed-4dd8-9442-1d0d6ff94734") : secret "dns-default-metrics-tls" not found
Sep 16 13:33:35.404930 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:35.404611    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls podName:b59d0af7-1225-42f1-9a74-29a6ad69b8df nodeName:}" failed. No retries permitted until 2026-09-16 13:33:43.404601439 +0000 UTC m=+51.088259813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls") pod "image-registry-74fd95df67-ttg94" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df") : secret "image-registry-tls" not found
Sep 16 13:33:38.027581 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.027549    2580 generic.go:356] "Generic (PLEG): container finished" podID="46a9e2e1-e39a-490d-a9ad-e919b33157dd" containerID="1ba69765a1ab2ada2746db88d712a09dcc68d80eedb2763ecdc7c5b5e3fca406" exitCode=0
Sep 16 13:33:38.027946 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.027591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerDied","Data":"1ba69765a1ab2ada2746db88d712a09dcc68d80eedb2763ecdc7c5b5e3fca406"}
Sep 16 13:33:38.031734 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.031369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc" event={"ID":"5769d743-ff5f-4061-adf6-d555412d32b1","Type":"ContainerStarted","Data":"62c9186636be50355bec9a0d329c2faa348dc7e39110d44e0d5e92b246fa83df"}
Sep 16 13:33:38.033124 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.033069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-snmh9" event={"ID":"493adda8-df96-418e-8f5d-b0b611d13187","Type":"ContainerStarted","Data":"42386b45f6a9fce124be9e98caf0f1fccf3e18423e61f4732b8cf1c797d13e93"}
Sep 16 13:33:38.034712 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.034673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg" event={"ID":"9fa7eea5-6a00-4cc2-9860-359ca3284c14","Type":"ContainerStarted","Data":"52b6ab06458a297a3f8b9d69eeb2137483c40560fec20cf5ae03549ce3d3c5f5"}
Sep 16 13:33:38.036202 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.036180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" event={"ID":"4f237f6d-0a4f-4559-b2fb-256e9598a755","Type":"ContainerStarted","Data":"fc844846962ac35eeb165440fe1a614c0e4a8f4f5097a3530bb540db5285e151"}
Sep 16 13:33:38.061582 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.061506    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:38.061848 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.061806    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc9c97b-7wqzc" podStartSLOduration=32.674727168 podStartE2EDuration="42.06178645s" totalImagesPullingTime="9.387059282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:38.061137639 +0000 UTC m=+45.744796044" watchObservedRunningTime="2026-09-16 13:33:38.06178645 +0000 UTC m=+45.745444843"
Sep 16 13:33:38.063587 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.063560    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg"
Sep 16 13:33:38.078276 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.078232    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-snmh9" podStartSLOduration=1.453260183 podStartE2EDuration="11.078207036s" totalImagesPullingTime="9.624946853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:38.077811592 +0000 UTC m=+45.761469986" watchObservedRunningTime="2026-09-16 13:33:38.078207036 +0000 UTC m=+45.761865431"
Sep 16 13:33:38.094477 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:38.093670    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54d887877d-h6brg" podStartSLOduration=32.724867259 podStartE2EDuration="42.093653362s" totalImagesPullingTime="9.368786103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:38.092478869 +0000 UTC m=+45.776137266" watchObservedRunningTime="2026-09-16 13:33:38.093653362 +0000 UTC m=+45.777311757"
Sep 16 13:33:39.041024 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:39.040992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" event={"ID":"4f237f6d-0a4f-4559-b2fb-256e9598a755","Type":"ContainerStarted","Data":"4e702c5ed9bfe290fc0d3dbf5d0939c2973291ebfd22aa8d5a38439f6a696ffa"}
Sep 16 13:33:39.041440 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:39.041041    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" event={"ID":"4f237f6d-0a4f-4559-b2fb-256e9598a755","Type":"ContainerStarted","Data":"6625e327a8116b1947dd6c5e20fa1fcd07dfcda4c277a36b6e2ffc9467cd8878"}
Sep 16 13:33:39.044192 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:39.044165    2580 generic.go:356] "Generic (PLEG): container finished" podID="46a9e2e1-e39a-490d-a9ad-e919b33157dd" containerID="c50f3cf58ab82072c4dfd0c7ea82f1aebf27319f9f1091385dea107104d0e337" exitCode=0
Sep 16 13:33:39.044316 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:39.044204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerDied","Data":"c50f3cf58ab82072c4dfd0c7ea82f1aebf27319f9f1091385dea107104d0e337"}
Sep 16 13:33:39.058277 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:39.057741    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" podStartSLOduration=33.664409422 podStartE2EDuration="43.057722674s" totalImagesPullingTime="9.393313252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:39.056948889 +0000 UTC m=+46.740607283" watchObservedRunningTime="2026-09-16 13:33:39.057722674 +0000 UTC m=+46.741381069"
Sep 16 13:33:40.051736 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:40.051693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" event={"ID":"46a9e2e1-e39a-490d-a9ad-e919b33157dd","Type":"ContainerStarted","Data":"4804b9523197a9c79b50fbc62f383a8db2723a6eac14dbbbf8e7413119be27dc"}
Sep 16 13:33:40.073672 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:40.073619    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jlm7v" podStartSLOduration=15.864192586 podStartE2EDuration="47.073602176s" totalImagesPullingTime="31.20940959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:33:40.072512998 +0000 UTC m=+47.756171391" watchObservedRunningTime="2026-09-16 13:33:40.073602176 +0000 UTC m=+47.757260568"
Sep 16 13:33:43.456543 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:43.456497    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:43.456959 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:43.456568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:43.456959 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:43.456668    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:33:43.456959 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:43.456752    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls podName:d56af732-9bed-4dd8-9442-1d0d6ff94734 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:59.456732987 +0000 UTC m=+67.140391370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls") pod "dns-default-xft82" (UID: "d56af732-9bed-4dd8-9442-1d0d6ff94734") : secret "dns-default-metrics-tls" not found
Sep 16 13:33:43.456959 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:43.456672    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:33:43.456959 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:43.456777    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74fd95df67-ttg94: secret "image-registry-tls" not found
Sep 16 13:33:43.456959 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:43.456835    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls podName:b59d0af7-1225-42f1-9a74-29a6ad69b8df nodeName:}" failed. No retries permitted until 2026-09-16 13:33:59.456821431 +0000 UTC m=+67.140479817 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls") pod "image-registry-74fd95df67-ttg94" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df") : secret "image-registry-tls" not found
Sep 16 13:33:52.985237 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:52.985210    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5vh56"
Sep 16 13:33:58.474274 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.474233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:33:58.476416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.476395    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 13:33:58.484892 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:58.484875    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 13:33:58.484959 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:58.484949    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs podName:6ea9a2a4-11ee-4027-affe-da9400ccd248 nodeName:}" failed. No retries permitted until 2026-09-16 13:35:02.484934037 +0000 UTC m=+130.168592423 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs") pod "network-metrics-daemon-wszjx" (UID: "6ea9a2a4-11ee-4027-affe-da9400ccd248") : secret "metrics-daemon-secret" not found
Sep 16 13:33:58.575553 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.575504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:58.577744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.577727    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 13:33:58.589284 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.589255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/78390bc8-31f3-4f46-81cb-08992be7587b-original-pull-secret\") pod \"global-pull-secret-syncer-sxq2j\" (UID: \"78390bc8-31f3-4f46-81cb-08992be7587b\") " pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:58.676333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.676288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:58.678514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.678496    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 13:33:58.688914 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.688879    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 13:33:58.699807 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.699775    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-974v4\" (UniqueName: \"kubernetes.io/projected/3215ed35-059b-46bb-b6ba-15c279a68841-kube-api-access-974v4\") pod \"network-check-target-5655w\" (UID: \"3215ed35-059b-46bb-b6ba-15c279a68841\") " pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:58.703848 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.703825    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wvdp2"]
Sep 16 13:33:58.706854 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.706837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.708718 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.708695    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 13:33:58.708971 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.708954    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 13:33:58.708971 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.708967    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 13:33:58.709364 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.709345    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sjsnp\""
Sep 16 13:33:58.709421 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.709349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 13:33:58.717641 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.717624    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvdp2"]
Sep 16 13:33:58.719138 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.719120    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-69tl6\""
Sep 16 13:33:58.726073 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.726022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sxq2j"
Sep 16 13:33:58.727408 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.727388    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:33:58.777956 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.777667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad0014ee-7144-46dc-8d4f-4fe57da43308-data-volume\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.777956 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.777716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.777956 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.777795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.777956 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.777831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgqhh\" (UniqueName: \"kubernetes.io/projected/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-api-access-hgqhh\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.777956 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.777862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad0014ee-7144-46dc-8d4f-4fe57da43308-crio-socket\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.878365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.878326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad0014ee-7144-46dc-8d4f-4fe57da43308-crio-socket\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.878559 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.878398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad0014ee-7144-46dc-8d4f-4fe57da43308-data-volume\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.878559 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.878414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.878559 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.878455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.878559 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.878468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad0014ee-7144-46dc-8d4f-4fe57da43308-crio-socket\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.878559 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.878474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgqhh\" (UniqueName: \"kubernetes.io/projected/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-api-access-hgqhh\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.878794 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:58.878579    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:33:58.878794 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:58.878578    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:33:58.878794 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:58.878683    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:59.378650779 +0000 UTC m=+67.062309174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : configmap "kube-rbac-proxy" not found
Sep 16 13:33:58.878794 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:58.878753    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:33:59.378735019 +0000 UTC m=+67.062393398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:33:58.879009 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.878812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad0014ee-7144-46dc-8d4f-4fe57da43308-data-volume\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.891698 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.891672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgqhh\" (UniqueName: \"kubernetes.io/projected/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-api-access-hgqhh\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:58.998772 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:58.998698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5655w"]
Sep 16 13:33:59.000598 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:59.000562    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sxq2j"]
Sep 16 13:33:59.002488 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:33:59.002464    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3215ed35_059b_46bb_b6ba_15c279a68841.slice/crio-f38c137056ccdeb03e2ec9144dff5c80d1fa4a6592b140049d1129306352fac0 WatchSource:0}: Error finding container f38c137056ccdeb03e2ec9144dff5c80d1fa4a6592b140049d1129306352fac0: Status 404 returned error can't find the container with id f38c137056ccdeb03e2ec9144dff5c80d1fa4a6592b140049d1129306352fac0
Sep 16 13:33:59.004011 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:33:59.003987    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78390bc8_31f3_4f46_81cb_08992be7587b.slice/crio-4e1b86ad7c8416242aeb2ab83972ad7201188c3fbe4e7caf69ec5a222fe9fc6f WatchSource:0}: Error finding container 4e1b86ad7c8416242aeb2ab83972ad7201188c3fbe4e7caf69ec5a222fe9fc6f: Status 404 returned error can't find the container with id 4e1b86ad7c8416242aeb2ab83972ad7201188c3fbe4e7caf69ec5a222fe9fc6f
Sep 16 13:33:59.109143 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:59.109103    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5655w" event={"ID":"3215ed35-059b-46bb-b6ba-15c279a68841","Type":"ContainerStarted","Data":"f38c137056ccdeb03e2ec9144dff5c80d1fa4a6592b140049d1129306352fac0"}
Sep 16 13:33:59.110205 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:59.110175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sxq2j" event={"ID":"78390bc8-31f3-4f46-81cb-08992be7587b","Type":"ContainerStarted","Data":"4e1b86ad7c8416242aeb2ab83972ad7201188c3fbe4e7caf69ec5a222fe9fc6f"}
Sep 16 13:33:59.383185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:59.383145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:59.383345 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:59.383204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:33:59.383345 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.383268    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:33:59.383345 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.383297    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:33:59.383345 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.383321    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:00.383307406 +0000 UTC m=+68.066965790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : configmap "kube-rbac-proxy" not found
Sep 16 13:33:59.383485 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.383353    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:00.383339982 +0000 UTC m=+68.066998357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:33:59.484198 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:59.484161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:33:59.484570 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:33:59.484231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:33:59.484570 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.484317    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 13:33:59.484570 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.484319    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 13:33:59.484570 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.484338    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74fd95df67-ttg94: secret "image-registry-tls" not found
Sep 16 13:33:59.484570 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.484371    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls podName:d56af732-9bed-4dd8-9442-1d0d6ff94734 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:31.484357347 +0000 UTC m=+99.168015735 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls") pod "dns-default-xft82" (UID: "d56af732-9bed-4dd8-9442-1d0d6ff94734") : secret "dns-default-metrics-tls" not found
Sep 16 13:33:59.484570 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:33:59.484386    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls podName:b59d0af7-1225-42f1-9a74-29a6ad69b8df nodeName:}" failed. No retries permitted until 2026-09-16 13:34:31.48437998 +0000 UTC m=+99.168038352 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls") pod "image-registry-74fd95df67-ttg94" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df") : secret "image-registry-tls" not found
Sep 16 13:34:00.391538 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:00.391496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:00.391733 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:00.391567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:00.391733 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:00.391671    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:34:00.391848 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:00.391739    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:02.391724129 +0000 UTC m=+70.075382505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:34:00.391848 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:00.391677    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:34:00.391848 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:00.391840    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:02.391824592 +0000 UTC m=+70.075482964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : configmap "kube-rbac-proxy" not found
Sep 16 13:34:00.855162 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:00.855057    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vcjzc_6a28437e-71de-45c2-8064-02e2d89aa324/dns-node-resolver/0.log"
Sep 16 13:34:01.455922 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:01.455877    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h4jzc_ed95382d-1fdc-40dd-a557-b20b2e4cb038/node-ca/0.log"
Sep 16 13:34:02.408225 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:02.408179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:02.408646 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:02.408319    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:34:02.408646 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:02.408365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:02.408646 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:02.408397    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:06.408375093 +0000 UTC m=+74.092033478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : configmap "kube-rbac-proxy" not found
Sep 16 13:34:02.408646 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:02.408463    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:34:02.408646 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:02.408518    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:06.408506697 +0000 UTC m=+74.092165218 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:34:02.458145 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:02.458109    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-snmh9_493adda8-df96-418e-8f5d-b0b611d13187/serve-healthcheck-canary/0.log"
Sep 16 13:34:05.132049 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:05.132014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5655w" event={"ID":"3215ed35-059b-46bb-b6ba-15c279a68841","Type":"ContainerStarted","Data":"3ab96c5129a51e0efb1675506ed37896c4d8d6f91d45e12bad1ce401b75ce69f"}
Sep 16 13:34:05.132543 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:05.132129    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:34:05.133576 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:05.133549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sxq2j" event={"ID":"78390bc8-31f3-4f46-81cb-08992be7587b","Type":"ContainerStarted","Data":"610f0d2de9b6fb3cfa54db32d9b48f8d69debe3f20fe301e3264d4219346f62e"}
Sep 16 13:34:05.149535 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:05.149491    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5655w" podStartSLOduration=65.953823839 podStartE2EDuration="1m11.149477943s" totalImagesPullingTime="5.195654104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:34:05.148283909 +0000 UTC m=+72.831942302" watchObservedRunningTime="2026-09-16 13:34:05.149477943 +0000 UTC m=+72.833136393"
Sep 16 13:34:05.162189 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:05.162147    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sxq2j" podStartSLOduration=65.734354882 podStartE2EDuration="1m11.162133218s" totalImagesPullingTime="5.427778336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:34:05.161769981 +0000 UTC m=+72.845428374" watchObservedRunningTime="2026-09-16 13:34:05.162133218 +0000 UTC m=+72.845791606"
Sep 16 13:34:06.439129 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:06.439088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:06.439585 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:06.439162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:06.439585 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:06.439243    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 13:34:06.439585 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:06.439333    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:14.43931428 +0000 UTC m=+82.122972652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : secret "insights-runtime-extractor-tls" not found
Sep 16 13:34:06.439585 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:06.439249    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 13:34:06.439585 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:06.439417    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm podName:ad0014ee-7144-46dc-8d4f-4fe57da43308 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:14.439398398 +0000 UTC m=+82.123056782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wvdp2" (UID: "ad0014ee-7144-46dc-8d4f-4fe57da43308") : configmap "kube-rbac-proxy" not found
Sep 16 13:34:14.504165 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:14.504124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:14.504617 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:14.504295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:14.504859 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:14.504836    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad0014ee-7144-46dc-8d4f-4fe57da43308-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:14.506483 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:14.506460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad0014ee-7144-46dc-8d4f-4fe57da43308-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wvdp2\" (UID: \"ad0014ee-7144-46dc-8d4f-4fe57da43308\") " pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:14.615964 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:14.615916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wvdp2"
Sep 16 13:34:14.883322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:14.883290    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wvdp2"]
Sep 16 13:34:14.886698 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:34:14.886663    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0014ee_7144_46dc_8d4f_4fe57da43308.slice/crio-401bede0eec34c543b31d676380aefa2164831c8e1f575fe44573e521f86790e WatchSource:0}: Error finding container 401bede0eec34c543b31d676380aefa2164831c8e1f575fe44573e521f86790e: Status 404 returned error can't find the container with id 401bede0eec34c543b31d676380aefa2164831c8e1f575fe44573e521f86790e
Sep 16 13:34:15.165498 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:15.165365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvdp2" event={"ID":"ad0014ee-7144-46dc-8d4f-4fe57da43308","Type":"ContainerStarted","Data":"038e7a130e1608992cc7a4b0492bc24057d16065165090f6d5abe36210b90586"}
Sep 16 13:34:15.165498 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:15.165421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvdp2" event={"ID":"ad0014ee-7144-46dc-8d4f-4fe57da43308","Type":"ContainerStarted","Data":"401bede0eec34c543b31d676380aefa2164831c8e1f575fe44573e521f86790e"}
Sep 16 13:34:16.170297 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:16.170266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvdp2" event={"ID":"ad0014ee-7144-46dc-8d4f-4fe57da43308","Type":"ContainerStarted","Data":"edd9edcd0b55806c9fb68d8681d7eed21e2fb639176002201d3fb8c9feddd72e"}
Sep 16 13:34:18.178563 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:18.178519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wvdp2" event={"ID":"ad0014ee-7144-46dc-8d4f-4fe57da43308","Type":"ContainerStarted","Data":"b22ecd06fb3a0dd7d59d0980ca9cd6729bd460f6235327308f0b214cbc3901dc"}
Sep 16 13:34:18.195364 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:18.195311    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wvdp2" podStartSLOduration=17.934874514 podStartE2EDuration="20.195296221s" totalImagesPullingTime="2.260421707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 13:33:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:34:18.193867806 +0000 UTC m=+85.877526200" watchObservedRunningTime="2026-09-16 13:34:18.195296221 +0000 UTC m=+85.878954615"
Sep 16 13:34:31.534701 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.534656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:34:31.535113 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.534752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:34:31.537030 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.537003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d56af732-9bed-4dd8-9442-1d0d6ff94734-metrics-tls\") pod \"dns-default-xft82\" (UID: \"d56af732-9bed-4dd8-9442-1d0d6ff94734\") " pod="openshift-dns/dns-default-xft82"
Sep 16 13:34:31.537139 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.537085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"image-registry-74fd95df67-ttg94\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") " pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:34:31.558753 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.558729    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c5bs9\""
Sep 16 13:34:31.567323 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.567309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:34:31.569178 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.569163    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nnkqw\""
Sep 16 13:34:31.577532 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.577516    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xft82"
Sep 16 13:34:31.840185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.840112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74fd95df67-ttg94"]
Sep 16 13:34:31.844473 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:34:31.844445    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59d0af7_1225_42f1_9a74_29a6ad69b8df.slice/crio-9b785655484f65e578a10cc5f528ae8198198cc307e586c84d658dd09dae1bec WatchSource:0}: Error finding container 9b785655484f65e578a10cc5f528ae8198198cc307e586c84d658dd09dae1bec: Status 404 returned error can't find the container with id 9b785655484f65e578a10cc5f528ae8198198cc307e586c84d658dd09dae1bec
Sep 16 13:34:31.856564 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:31.856530    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xft82"]
Sep 16 13:34:31.859783 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:34:31.859758    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd56af732_9bed_4dd8_9442_1d0d6ff94734.slice/crio-7837c2957dc5ef0c3a89118a936526ef55a2797ca3408fa5663aa87f4c22cdf4 WatchSource:0}: Error finding container 7837c2957dc5ef0c3a89118a936526ef55a2797ca3408fa5663aa87f4c22cdf4: Status 404 returned error can't find the container with id 7837c2957dc5ef0c3a89118a936526ef55a2797ca3408fa5663aa87f4c22cdf4
Sep 16 13:34:32.222450 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:32.222413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xft82" event={"ID":"d56af732-9bed-4dd8-9442-1d0d6ff94734","Type":"ContainerStarted","Data":"7837c2957dc5ef0c3a89118a936526ef55a2797ca3408fa5663aa87f4c22cdf4"}
Sep 16 13:34:32.224174 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:32.224140    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74fd95df67-ttg94" event={"ID":"b59d0af7-1225-42f1-9a74-29a6ad69b8df","Type":"ContainerStarted","Data":"1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f"}
Sep 16 13:34:32.224315 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:32.224183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74fd95df67-ttg94" event={"ID":"b59d0af7-1225-42f1-9a74-29a6ad69b8df","Type":"ContainerStarted","Data":"9b785655484f65e578a10cc5f528ae8198198cc307e586c84d658dd09dae1bec"}
Sep 16 13:34:32.224315 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:32.224285    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:34:32.245482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:32.245407    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74fd95df67-ttg94" podStartSLOduration=98.245390052 podStartE2EDuration="1m38.245390052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:34:32.244350616 +0000 UTC m=+99.928009076" watchObservedRunningTime="2026-09-16 13:34:32.245390052 +0000 UTC m=+99.929048445"
Sep 16 13:34:34.232693 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:34.232660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xft82" event={"ID":"d56af732-9bed-4dd8-9442-1d0d6ff94734","Type":"ContainerStarted","Data":"773f6d9d66002c5fd1a71be20b04a4a938ca045ea3c170bb07b834af66836a63"}
Sep 16 13:34:34.233092 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:34.232712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xft82" event={"ID":"d56af732-9bed-4dd8-9442-1d0d6ff94734","Type":"ContainerStarted","Data":"a4a1a2534910369c7ddd16d3c9834ddfaeb07fd8bf02d5c8804eb5a8ccdc2d9d"}
Sep 16 13:34:34.233092 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:34.232786    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xft82"
Sep 16 13:34:34.250077 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:34.250025    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xft82" podStartSLOduration=65.234021933 podStartE2EDuration="1m7.250012528s" totalImagesPullingTime="2.015990595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:34:34.24842294 +0000 UTC m=+101.932081334" watchObservedRunningTime="2026-09-16 13:34:34.250012528 +0000 UTC m=+101.933670941"
Sep 16 13:34:36.139336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.139304    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5655w"
Sep 16 13:34:36.618237 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.618153    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h9fxb"]
Sep 16 13:34:36.620396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.620378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.622218 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.622193    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 13:34:36.622480 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.622464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 13:34:36.622742 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.622726    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 13:34:36.622869 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.622797    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bdhnc\""
Sep 16 13:34:36.622993 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.622956    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 13:34:36.623276 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.623261    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 13:34:36.623464 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.623447    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 13:34:36.672567 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-wtmp\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672567 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672737 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm9wt\" (UniqueName: \"kubernetes.io/projected/d0abaff1-c5a8-4893-9303-2aa90e7af317-kube-api-access-mm9wt\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672737 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-sys\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672737 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-textfile\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672867 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0abaff1-c5a8-4893-9303-2aa90e7af317-metrics-client-ca\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672867 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-accelerators-collector-config\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672867 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-root\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.672867 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.672858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-tls\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774043 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm9wt\" (UniqueName: \"kubernetes.io/projected/d0abaff1-c5a8-4893-9303-2aa90e7af317-kube-api-access-mm9wt\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774043 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-sys\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-textfile\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0abaff1-c5a8-4893-9303-2aa90e7af317-metrics-client-ca\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-accelerators-collector-config\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-root\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-sys\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-root\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-tls\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774227 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-wtmp\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774602 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774602 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:36.774308    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 16 13:34:36.774602 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:34:36.774366    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-tls podName:d0abaff1-c5a8-4893-9303-2aa90e7af317 nodeName:}" failed. No retries permitted until 2026-09-16 13:34:37.274347385 +0000 UTC m=+104.958005763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-tls") pod "node-exporter-h9fxb" (UID: "d0abaff1-c5a8-4893-9303-2aa90e7af317") : secret "node-exporter-tls" not found
Sep 16 13:34:36.774602 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-wtmp\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774602 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-textfile\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774846 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-accelerators-collector-config\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.774846 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.774759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0abaff1-c5a8-4893-9303-2aa90e7af317-metrics-client-ca\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.776451 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.776434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:36.781935 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:36.781889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm9wt\" (UniqueName: \"kubernetes.io/projected/d0abaff1-c5a8-4893-9303-2aa90e7af317-kube-api-access-mm9wt\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:37.277743 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:37.277708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-tls\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:37.279931 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:37.279911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0abaff1-c5a8-4893-9303-2aa90e7af317-node-exporter-tls\") pod \"node-exporter-h9fxb\" (UID: \"d0abaff1-c5a8-4893-9303-2aa90e7af317\") " pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:37.529297 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:37.529229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h9fxb"
Sep 16 13:34:37.539053 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:34:37.539031    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0abaff1_c5a8_4893_9303_2aa90e7af317.slice/crio-21c8063e434f0f6a0c8d2d7cb31047c746fb9e309c405f7c6e823cc1e2a581c7 WatchSource:0}: Error finding container 21c8063e434f0f6a0c8d2d7cb31047c746fb9e309c405f7c6e823cc1e2a581c7: Status 404 returned error can't find the container with id 21c8063e434f0f6a0c8d2d7cb31047c746fb9e309c405f7c6e823cc1e2a581c7
Sep 16 13:34:38.247744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:38.247698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h9fxb" event={"ID":"d0abaff1-c5a8-4893-9303-2aa90e7af317","Type":"ContainerStarted","Data":"21c8063e434f0f6a0c8d2d7cb31047c746fb9e309c405f7c6e823cc1e2a581c7"}
Sep 16 13:34:39.253435 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:39.253403    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0abaff1-c5a8-4893-9303-2aa90e7af317" containerID="c4cadd61fb53677b9e30806cb8bc8dc59d19e0d101e15a67f76497402bbc42f4" exitCode=0
Sep 16 13:34:39.253782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:39.253463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h9fxb" event={"ID":"d0abaff1-c5a8-4893-9303-2aa90e7af317","Type":"ContainerDied","Data":"c4cadd61fb53677b9e30806cb8bc8dc59d19e0d101e15a67f76497402bbc42f4"}
Sep 16 13:34:40.258532 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:40.258498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h9fxb" event={"ID":"d0abaff1-c5a8-4893-9303-2aa90e7af317","Type":"ContainerStarted","Data":"b38ed410de4ee5eaec7235948dfe7bf910845e222a08901a841e38aba66e8f59"}
Sep 16 13:34:40.258948 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:40.258577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h9fxb" event={"ID":"d0abaff1-c5a8-4893-9303-2aa90e7af317","Type":"ContainerStarted","Data":"b17d50a5702948bcac5007c5de719748f91846f2c4293d15fbfa2cb5344fe911"}
Sep 16 13:34:40.279410 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:40.279360    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h9fxb" podStartSLOduration=3.194650144 podStartE2EDuration="4.27934648s" totalImagesPullingTime="1.02228937s" totalInitContainerRuntime="62.406966ms" isStatefulPod=true podCreationTimestamp="2026-09-16 13:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:34:40.278788685 +0000 UTC m=+107.962447076" watchObservedRunningTime="2026-09-16 13:34:40.27934648 +0000 UTC m=+107.963004873"
Sep 16 13:34:42.488006 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:42.487971    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74fd95df67-ttg94"]
Sep 16 13:34:44.238193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:44.238159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xft82"
Sep 16 13:34:47.933851 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:47.933805    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" podUID="4f237f6d-0a4f-4559-b2fb-256e9598a755" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 13:34:52.492658 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:52.492632    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:34:57.934087 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:34:57.934046    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" podUID="4f237f6d-0a4f-4559-b2fb-256e9598a755" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 13:35:02.553909 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:02.553871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:35:02.556191 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:02.556169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ea9a2a4-11ee-4027-affe-da9400ccd248-metrics-certs\") pod \"network-metrics-daemon-wszjx\" (UID: \"6ea9a2a4-11ee-4027-affe-da9400ccd248\") " pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:35:02.611823 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:02.611793    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2cz7s\""
Sep 16 13:35:02.620325 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:02.620303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wszjx"
Sep 16 13:35:02.888154 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:02.888125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wszjx"]
Sep 16 13:35:02.891066 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:35:02.891027    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea9a2a4_11ee_4027_affe_da9400ccd248.slice/crio-16d64aed28bb1a9f51396d09655f78e28107241f467cb71e56e109ed8c2b5479 WatchSource:0}: Error finding container 16d64aed28bb1a9f51396d09655f78e28107241f467cb71e56e109ed8c2b5479: Status 404 returned error can't find the container with id 16d64aed28bb1a9f51396d09655f78e28107241f467cb71e56e109ed8c2b5479
Sep 16 13:35:03.329607 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:03.329527    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wszjx" event={"ID":"6ea9a2a4-11ee-4027-affe-da9400ccd248","Type":"ContainerStarted","Data":"16d64aed28bb1a9f51396d09655f78e28107241f467cb71e56e109ed8c2b5479"}
Sep 16 13:35:04.747212 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:04.747176    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-snmh9_493adda8-df96-418e-8f5d-b0b611d13187/serve-healthcheck-canary/0.log"
Sep 16 13:35:05.337451 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:05.337419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wszjx" event={"ID":"6ea9a2a4-11ee-4027-affe-da9400ccd248","Type":"ContainerStarted","Data":"580ae8bb6a455d1fb4e9f524602d395edfabd206e6332910308dfe2a484ffa61"}
Sep 16 13:35:05.337561 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:05.337476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wszjx" event={"ID":"6ea9a2a4-11ee-4027-affe-da9400ccd248","Type":"ContainerStarted","Data":"7b4b53b9f297c8b641745a026999df7ff2915c35a42ce13158a60ce5729e5089"}
Sep 16 13:35:05.353703 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:05.353618    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wszjx" podStartSLOduration=130.153917398 podStartE2EDuration="2m12.35360115s" totalImagesPullingTime="2.199683752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:35:05.353128422 +0000 UTC m=+133.036786815" watchObservedRunningTime="2026-09-16 13:35:05.35360115 +0000 UTC m=+133.037259544"
Sep 16 13:35:07.506537 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.506476    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74fd95df67-ttg94" podUID="b59d0af7-1225-42f1-9a74-29a6ad69b8df" containerName="registry" containerID="cri-o://1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f" gracePeriod=30
Sep 16 13:35:07.738922 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.738884    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:35:07.787888 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.787812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b59d0af7-1225-42f1-9a74-29a6ad69b8df-ca-trust-extracted\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.787888 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.787847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2stf\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-kube-api-access-m2stf\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.787888 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.787867    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-trusted-ca\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.787888 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.787885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-installation-pull-secrets\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.788221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.787920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.788221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.787964    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-certificates\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.788221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.787983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-bound-sa-token\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.788221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.788029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-image-registry-private-configuration\") pod \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\" (UID: \"b59d0af7-1225-42f1-9a74-29a6ad69b8df\") "
Sep 16 13:35:07.788448 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.788418    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:35:07.788537 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.788500    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:35:07.790415 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.790386    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-kube-api-access-m2stf" (OuterVolumeSpecName: "kube-api-access-m2stf") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "kube-api-access-m2stf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:35:07.790531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.790393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:35:07.790531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.790444    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:35:07.790531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.790496    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:35:07.790926 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.790874    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:35:07.796828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.796804    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b59d0af7-1225-42f1-9a74-29a6ad69b8df-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b59d0af7-1225-42f1-9a74-29a6ad69b8df" (UID: "b59d0af7-1225-42f1-9a74-29a6ad69b8df"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:35:07.888606 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888571    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-image-registry-private-configuration\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.888606 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888602    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b59d0af7-1225-42f1-9a74-29a6ad69b8df-ca-trust-extracted\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.888782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888617    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2stf\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-kube-api-access-m2stf\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.888782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888631    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-trusted-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.888782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888643    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b59d0af7-1225-42f1-9a74-29a6ad69b8df-installation-pull-secrets\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.888782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888654    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-tls\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.888782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888665    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b59d0af7-1225-42f1-9a74-29a6ad69b8df-registry-certificates\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.888782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.888676    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b59d0af7-1225-42f1-9a74-29a6ad69b8df-bound-sa-token\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:35:07.934396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.934361    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" podUID="4f237f6d-0a4f-4559-b2fb-256e9598a755" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 13:35:07.934516 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.934430    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49"
Sep 16 13:35:07.934957 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.934926    2580 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"4e702c5ed9bfe290fc0d3dbf5d0939c2973291ebfd22aa8d5a38439f6a696ffa"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 16 13:35:07.934994 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:07.934979    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" podUID="4f237f6d-0a4f-4559-b2fb-256e9598a755" containerName="service-proxy" containerID="cri-o://4e702c5ed9bfe290fc0d3dbf5d0939c2973291ebfd22aa8d5a38439f6a696ffa" gracePeriod=30
Sep 16 13:35:08.356118 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.356030    2580 generic.go:356] "Generic (PLEG): container finished" podID="4f237f6d-0a4f-4559-b2fb-256e9598a755" containerID="4e702c5ed9bfe290fc0d3dbf5d0939c2973291ebfd22aa8d5a38439f6a696ffa" exitCode=2
Sep 16 13:35:08.356118 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.356102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" event={"ID":"4f237f6d-0a4f-4559-b2fb-256e9598a755","Type":"ContainerDied","Data":"4e702c5ed9bfe290fc0d3dbf5d0939c2973291ebfd22aa8d5a38439f6a696ffa"}
Sep 16 13:35:08.356302 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.356143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f74495c67-55l49" event={"ID":"4f237f6d-0a4f-4559-b2fb-256e9598a755","Type":"ContainerStarted","Data":"011a104ef14cc25bf8b14e89bcd55c006d0e86ceea841ac7e8d35b6f9ddfcc85"}
Sep 16 13:35:08.357345 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.357319    2580 generic.go:356] "Generic (PLEG): container finished" podID="b59d0af7-1225-42f1-9a74-29a6ad69b8df" containerID="1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f" exitCode=0
Sep 16 13:35:08.357449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.357358    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74fd95df67-ttg94"
Sep 16 13:35:08.357449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.357373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74fd95df67-ttg94" event={"ID":"b59d0af7-1225-42f1-9a74-29a6ad69b8df","Type":"ContainerDied","Data":"1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f"}
Sep 16 13:35:08.357449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.357412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74fd95df67-ttg94" event={"ID":"b59d0af7-1225-42f1-9a74-29a6ad69b8df","Type":"ContainerDied","Data":"9b785655484f65e578a10cc5f528ae8198198cc307e586c84d658dd09dae1bec"}
Sep 16 13:35:08.357449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.357439    2580 scope.go:117] "RemoveContainer" containerID="1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f"
Sep 16 13:35:08.366162 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.366143    2580 scope.go:117] "RemoveContainer" containerID="1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f"
Sep 16 13:35:08.366422 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:35:08.366403    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f\": container with ID starting with 1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f not found: ID does not exist" containerID="1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f"
Sep 16 13:35:08.366504 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.366429    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f"} err="failed to get container status \"1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f\": rpc error: code = NotFound desc = could not find container \"1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f\": container with ID starting with 1a0468ca4c256501dcd2877a55a8b7d89df994bfffca9d1e95b56e117e4e3c0f not found: ID does not exist"
Sep 16 13:35:08.383236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.383214    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74fd95df67-ttg94"]
Sep 16 13:35:08.386560 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.386542    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74fd95df67-ttg94"]
Sep 16 13:35:08.801717 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:35:08.801678    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b59d0af7-1225-42f1-9a74-29a6ad69b8df" path="/var/lib/kubelet/pods/b59d0af7-1225-42f1-9a74-29a6ad69b8df/volumes"
Sep 16 13:37:52.662217 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:37:52.662184    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:37:52.669107 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:37:52.669088    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 13:37:52.672449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:37:52.672424    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:38:03.386803 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.386775    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"]
Sep 16 13:38:03.389318 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.386990    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b59d0af7-1225-42f1-9a74-29a6ad69b8df" containerName="registry"
Sep 16 13:38:03.389318 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.387001    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b59d0af7-1225-42f1-9a74-29a6ad69b8df" containerName="registry"
Sep 16 13:38:03.389318 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.387054    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b59d0af7-1225-42f1-9a74-29a6ad69b8df" containerName="registry"
Sep 16 13:38:03.390175 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.390160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.392676 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.392657    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gncgw\""
Sep 16 13:38:03.393205 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.393189    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 13:38:03.393274 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.393259    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 13:38:03.401232 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.401210    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"]
Sep 16 13:38:03.501047 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.501025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e50e32-f144-4a8a-bbde-78656e9320e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fs2xp\" (UID: \"49e50e32-f144-4a8a-bbde-78656e9320e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.501169 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.501055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fjcc\" (UniqueName: \"kubernetes.io/projected/49e50e32-f144-4a8a-bbde-78656e9320e7-kube-api-access-6fjcc\") pod \"openshift-lws-operator-fd8ccff4c-fs2xp\" (UID: \"49e50e32-f144-4a8a-bbde-78656e9320e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.601479 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.601450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e50e32-f144-4a8a-bbde-78656e9320e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fs2xp\" (UID: \"49e50e32-f144-4a8a-bbde-78656e9320e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.601588 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.601491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjcc\" (UniqueName: \"kubernetes.io/projected/49e50e32-f144-4a8a-bbde-78656e9320e7-kube-api-access-6fjcc\") pod \"openshift-lws-operator-fd8ccff4c-fs2xp\" (UID: \"49e50e32-f144-4a8a-bbde-78656e9320e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.601821 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.601801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e50e32-f144-4a8a-bbde-78656e9320e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fs2xp\" (UID: \"49e50e32-f144-4a8a-bbde-78656e9320e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.608637 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.608595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fjcc\" (UniqueName: \"kubernetes.io/projected/49e50e32-f144-4a8a-bbde-78656e9320e7-kube-api-access-6fjcc\") pod \"openshift-lws-operator-fd8ccff4c-fs2xp\" (UID: \"49e50e32-f144-4a8a-bbde-78656e9320e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.698558 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.698492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"
Sep 16 13:38:03.966138 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.966070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp"]
Sep 16 13:38:03.969169 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:38:03.969144    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49e50e32_f144_4a8a_bbde_78656e9320e7.slice/crio-3a4a19641e72ccfcbec53c1a8f72321de0d31ea4eb01b6a0db62bf8e2a0224f0 WatchSource:0}: Error finding container 3a4a19641e72ccfcbec53c1a8f72321de0d31ea4eb01b6a0db62bf8e2a0224f0: Status 404 returned error can't find the container with id 3a4a19641e72ccfcbec53c1a8f72321de0d31ea4eb01b6a0db62bf8e2a0224f0
Sep 16 13:38:03.970659 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:03.970639    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:38:04.908889 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:04.908849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp" event={"ID":"49e50e32-f144-4a8a-bbde-78656e9320e7","Type":"ContainerStarted","Data":"3a4a19641e72ccfcbec53c1a8f72321de0d31ea4eb01b6a0db62bf8e2a0224f0"}
Sep 16 13:38:06.916823 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:06.916784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp" event={"ID":"49e50e32-f144-4a8a-bbde-78656e9320e7","Type":"ContainerStarted","Data":"0a7b2c14b596b71beabf9d200a11cddbd83d26c018168158e07295e5fcb65977"}
Sep 16 13:38:06.930735 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:06.930679    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fs2xp" podStartSLOduration=1.534185639 podStartE2EDuration="3.930660819s" totalImagesPullingTime="2.39647518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:38:06.930101471 +0000 UTC m=+314.613759861" watchObservedRunningTime="2026-09-16 13:38:06.930660819 +0000 UTC m=+314.614319213"
Sep 16 13:38:22.115181 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.115146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"]
Sep 16 13:38:22.119296 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.119275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.121595 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.121571    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8t96b\""
Sep 16 13:38:22.121723 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.121604    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 13:38:22.121944 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.121919    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 13:38:22.122046 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.121950    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 13:38:22.122046 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.121927    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 13:38:22.132928 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.132891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"]
Sep 16 13:38:22.220217 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.220186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-tmp\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.220339 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.220222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-manifests\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.220339 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.220268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.220339 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.220306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-webhook-cert\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.220460 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.220378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf5ns\" (UniqueName: \"kubernetes.io/projected/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-kube-api-access-gf5ns\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.320916 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.320873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-tmp\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.321016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.320922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-manifests\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.321016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.320952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.321016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.320969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-webhook-cert\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.321016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.320993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf5ns\" (UniqueName: \"kubernetes.io/projected/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-kube-api-access-gf5ns\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.321356 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.321330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-manifests\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.323175 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.323153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-tmp\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.323384 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.323369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-webhook-cert\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.323442 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.323425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.333348 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.333329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf5ns\" (UniqueName: \"kubernetes.io/projected/6abfc89e-1c1e-4c68-84dd-f0cbd985a965-kube-api-access-gf5ns\") pod \"opendatahub-operator-controller-manager-6444576fdb-62lqn\" (UID: \"6abfc89e-1c1e-4c68-84dd-f0cbd985a965\") " pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.428741 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.428723    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:22.717972 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.717949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"]
Sep 16 13:38:22.720073 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:38:22.720040    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abfc89e_1c1e_4c68_84dd_f0cbd985a965.slice/crio-e69281dabe99939498d2641c550d5d258d976fa46fa3be7b09489250ab65de5f WatchSource:0}: Error finding container e69281dabe99939498d2641c550d5d258d976fa46fa3be7b09489250ab65de5f: Status 404 returned error can't find the container with id e69281dabe99939498d2641c550d5d258d976fa46fa3be7b09489250ab65de5f
Sep 16 13:38:22.966135 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:22.966101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn" event={"ID":"6abfc89e-1c1e-4c68-84dd-f0cbd985a965","Type":"ContainerStarted","Data":"e69281dabe99939498d2641c550d5d258d976fa46fa3be7b09489250ab65de5f"}
Sep 16 13:38:25.978053 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:25.978017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn" event={"ID":"6abfc89e-1c1e-4c68-84dd-f0cbd985a965","Type":"ContainerStarted","Data":"9464093573fb3d0b755e18d47c2335d4096035b1893d42305b7fdaaa78351089"}
Sep 16 13:38:26.984947 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:26.984919    2580 generic.go:356] "Generic (PLEG): container finished" podID="6abfc89e-1c1e-4c68-84dd-f0cbd985a965" containerID="9464093573fb3d0b755e18d47c2335d4096035b1893d42305b7fdaaa78351089" exitCode=0
Sep 16 13:38:26.985322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:26.984993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn" event={"ID":"6abfc89e-1c1e-4c68-84dd-f0cbd985a965","Type":"ContainerDied","Data":"9464093573fb3d0b755e18d47c2335d4096035b1893d42305b7fdaaa78351089"}
Sep 16 13:38:27.990676 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:27.990645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn" event={"ID":"6abfc89e-1c1e-4c68-84dd-f0cbd985a965","Type":"ContainerStarted","Data":"6ecd50d1925dce576b874a21582ae5791ca79a7164f033ddec8aa52c21ae0e25"}
Sep 16 13:38:27.991110 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:27.990786    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:28.012317 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:28.012261    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn" podStartSLOduration=1.788974943 podStartE2EDuration="6.012243202s" totalImagesPullingTime="3.170509261s" totalInitContainerRuntime="1.052758998s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:38:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:38:28.010437166 +0000 UTC m=+335.694095561" watchObservedRunningTime="2026-09-16 13:38:28.012243202 +0000 UTC m=+335.695901596"
Sep 16 13:38:38.996490 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:38.996456    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6444576fdb-62lqn"
Sep 16 13:38:41.506777 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.506747    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"]
Sep 16 13:38:41.515012 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.514994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.518983 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.518963    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dltcb\""
Sep 16 13:38:41.530217 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.530202    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 13:38:41.531619 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.531604    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 13:38:41.533703 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.533683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"]
Sep 16 13:38:41.540612 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.540594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 13:38:41.540874 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.540862    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 13:38:41.661954 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.661921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ce32683-d750-4f31-888c-3db9ef3e21f1-tmp\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.662084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.661961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2qs6\" (UniqueName: \"kubernetes.io/projected/8ce32683-d750-4f31-888c-3db9ef3e21f1-kube-api-access-c2qs6\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.662084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.662010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce32683-d750-4f31-888c-3db9ef3e21f1-tls-certs\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.762677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.762612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ce32683-d750-4f31-888c-3db9ef3e21f1-tmp\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.762677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.762646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2qs6\" (UniqueName: \"kubernetes.io/projected/8ce32683-d750-4f31-888c-3db9ef3e21f1-kube-api-access-c2qs6\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.762677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.762669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce32683-d750-4f31-888c-3db9ef3e21f1-tls-certs\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.764889 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.764857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ce32683-d750-4f31-888c-3db9ef3e21f1-tmp\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.765012 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.764993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce32683-d750-4f31-888c-3db9ef3e21f1-tls-certs\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.770670 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.770643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2qs6\" (UniqueName: \"kubernetes.io/projected/8ce32683-d750-4f31-888c-3db9ef3e21f1-kube-api-access-c2qs6\") pod \"kube-auth-proxy-68bbd697f4-l4hhd\" (UID: \"8ce32683-d750-4f31-888c-3db9ef3e21f1\") " pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:41.823116 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:41.823094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"
Sep 16 13:38:42.092490 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.092419    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd"]
Sep 16 13:38:42.095639 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:38:42.095611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce32683_d750_4f31_888c_3db9ef3e21f1.slice/crio-8aa4dd6e2d8cf0c02fb55392028a1e5c6b4a71521d33fde47803fe49a201d15f WatchSource:0}: Error finding container 8aa4dd6e2d8cf0c02fb55392028a1e5c6b4a71521d33fde47803fe49a201d15f: Status 404 returned error can't find the container with id 8aa4dd6e2d8cf0c02fb55392028a1e5c6b4a71521d33fde47803fe49a201d15f
Sep 16 13:38:42.232806 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.232776    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"]
Sep 16 13:38:42.236910 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.236886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.239161 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.239138    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 13:38:42.239254 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.239140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 13:38:42.239617 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.239601    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 13:38:42.239694 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.239638    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ccjkg\""
Sep 16 13:38:42.245604 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.245586    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"]
Sep 16 13:38:42.367630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.367570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d1d7e2f-251e-473b-afbd-103c162fc02b-metrics-cert\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.367630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.367615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxw7r\" (UniqueName: \"kubernetes.io/projected/3d1d7e2f-251e-473b-afbd-103c162fc02b-kube-api-access-fxw7r\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.367770 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.367633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d1d7e2f-251e-473b-afbd-103c162fc02b-cert\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.367770 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.367703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3d1d7e2f-251e-473b-afbd-103c162fc02b-manager-config\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.468088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.468059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d1d7e2f-251e-473b-afbd-103c162fc02b-metrics-cert\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.468201 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.468118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxw7r\" (UniqueName: \"kubernetes.io/projected/3d1d7e2f-251e-473b-afbd-103c162fc02b-kube-api-access-fxw7r\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.468201 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.468157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d1d7e2f-251e-473b-afbd-103c162fc02b-cert\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.468321 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.468204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3d1d7e2f-251e-473b-afbd-103c162fc02b-manager-config\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.468838 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.468817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3d1d7e2f-251e-473b-afbd-103c162fc02b-manager-config\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.470417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.470393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d1d7e2f-251e-473b-afbd-103c162fc02b-metrics-cert\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.470501 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.470427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d1d7e2f-251e-473b-afbd-103c162fc02b-cert\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.478987 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.478971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxw7r\" (UniqueName: \"kubernetes.io/projected/3d1d7e2f-251e-473b-afbd-103c162fc02b-kube-api-access-fxw7r\") pod \"lws-controller-manager-8878d4c7-q6kzw\" (UID: \"3d1d7e2f-251e-473b-afbd-103c162fc02b\") " pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.545945 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.545924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:42.820586 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:42.820532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"]
Sep 16 13:38:42.823994 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:38:42.823952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1d7e2f_251e_473b_afbd_103c162fc02b.slice/crio-cfb1721df62f2e4871f018ab6b3abd06b848718de9c8b8a4f09a43056e5bf874 WatchSource:0}: Error finding container cfb1721df62f2e4871f018ab6b3abd06b848718de9c8b8a4f09a43056e5bf874: Status 404 returned error can't find the container with id cfb1721df62f2e4871f018ab6b3abd06b848718de9c8b8a4f09a43056e5bf874
Sep 16 13:38:43.041796 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:43.041741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw" event={"ID":"3d1d7e2f-251e-473b-afbd-103c162fc02b","Type":"ContainerStarted","Data":"cfb1721df62f2e4871f018ab6b3abd06b848718de9c8b8a4f09a43056e5bf874"}
Sep 16 13:38:43.043608 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:43.043526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd" event={"ID":"8ce32683-d750-4f31-888c-3db9ef3e21f1","Type":"ContainerStarted","Data":"8aa4dd6e2d8cf0c02fb55392028a1e5c6b4a71521d33fde47803fe49a201d15f"}
Sep 16 13:38:45.053399 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:45.053317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd" event={"ID":"8ce32683-d750-4f31-888c-3db9ef3e21f1","Type":"ContainerStarted","Data":"e9d9919670fe140bc22951393e812dc7861e65fe6a722228284669419fc6f0a9"}
Sep 16 13:38:45.068795 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:45.068738    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68bbd697f4-l4hhd" podStartSLOduration=1.222647731 podStartE2EDuration="4.068722329s" totalImagesPullingTime="2.846074598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:38:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:38:45.067480131 +0000 UTC m=+352.751138523" watchObservedRunningTime="2026-09-16 13:38:45.068722329 +0000 UTC m=+352.752380736"
Sep 16 13:38:46.059098 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:46.059007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw" event={"ID":"3d1d7e2f-251e-473b-afbd-103c162fc02b","Type":"ContainerStarted","Data":"8722fe1a63cf12db64256bee08cb3c726e59b7b61cb1ead050aff6402309ac56"}
Sep 16 13:38:46.075215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:46.075169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw" podStartSLOduration=1.172425356 podStartE2EDuration="4.07515333s" totalImagesPullingTime="2.902727974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:38:46.074082194 +0000 UTC m=+353.757740589" watchObservedRunningTime="2026-09-16 13:38:46.07515333 +0000 UTC m=+353.758811725"
Sep 16 13:38:47.062630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:47.062597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:38:58.067733 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:38:58.067699    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8878d4c7-q6kzw"
Sep 16 13:40:25.946200 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.946126    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"]
Sep 16 13:40:25.949106 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.949089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:25.952444 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.952421    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 13:40:25.952444 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.952438    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 13:40:25.952444 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.952454    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 13:40:25.952691 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.952421    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 13:40:25.952826 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.952813    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fz48k\""
Sep 16 13:40:25.964933 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:25.964914    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"]
Sep 16 13:40:26.061137 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.061102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.061137 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.061144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.061289 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.061183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brwbw\" (UniqueName: \"kubernetes.io/projected/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-kube-api-access-brwbw\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.161957 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.161932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.162074 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.161984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brwbw\" (UniqueName: \"kubernetes.io/projected/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-kube-api-access-brwbw\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.162074 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.162049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.162513 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.162493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.164298 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.164279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.169369 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.169347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brwbw\" (UniqueName: \"kubernetes.io/projected/534957fb-aaa7-4f63-8ae1-d2f55a1869b7-kube-api-access-brwbw\") pod \"kuadrant-console-plugin-fbdd4f445-cjj6h\" (UID: \"534957fb-aaa7-4f63-8ae1-d2f55a1869b7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.257446 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.257395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"
Sep 16 13:40:26.523086 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:26.523005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h"]
Sep 16 13:40:26.525505 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:40:26.525479    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod534957fb_aaa7_4f63_8ae1_d2f55a1869b7.slice/crio-29991bd340aca89e210c77f81a2d9af3fb743dae8ff528cdbcd668524dfc0606 WatchSource:0}: Error finding container 29991bd340aca89e210c77f81a2d9af3fb743dae8ff528cdbcd668524dfc0606: Status 404 returned error can't find the container with id 29991bd340aca89e210c77f81a2d9af3fb743dae8ff528cdbcd668524dfc0606
Sep 16 13:40:27.412518 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:27.412485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h" event={"ID":"534957fb-aaa7-4f63-8ae1-d2f55a1869b7","Type":"ContainerStarted","Data":"29991bd340aca89e210c77f81a2d9af3fb743dae8ff528cdbcd668524dfc0606"}
Sep 16 13:40:33.440692 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:33.440656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h" event={"ID":"534957fb-aaa7-4f63-8ae1-d2f55a1869b7","Type":"ContainerStarted","Data":"5cdd4dd5084fac676ef55e24b03022c1c00a122db754febeb8ae7c0198fb7f3b"}
Sep 16 13:40:33.455217 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:40:33.455174    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cjj6h" podStartSLOduration=2.577116787 podStartE2EDuration="8.455160618s" totalImagesPullingTime="5.878043831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:40:33.454054548 +0000 UTC m=+461.137712953" watchObservedRunningTime="2026-09-16 13:40:33.455160618 +0000 UTC m=+461.138819011"
Sep 16 13:41:25.393260 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.391554    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5chsz"]
Sep 16 13:41:25.395381 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.395350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz"
Sep 16 13:41:25.397259 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.397236    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2bgzw\""
Sep 16 13:41:25.399857 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.399831    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5chsz"]
Sep 16 13:41:25.511306 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.511272    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-lkkv2"]
Sep 16 13:41:25.514117 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.514098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lkkv2"
Sep 16 13:41:25.519239 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.519217    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lkkv2"]
Sep 16 13:41:25.554667 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.554631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc8kl\" (UniqueName: \"kubernetes.io/projected/c2e64835-1691-4481-9d8b-e2a49ca7ba3c-kube-api-access-xc8kl\") pod \"authorino-7bcfd8f9cd-5chsz\" (UID: \"c2e64835-1691-4481-9d8b-e2a49ca7ba3c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz"
Sep 16 13:41:25.655173 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.655122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc8kl\" (UniqueName: \"kubernetes.io/projected/c2e64835-1691-4481-9d8b-e2a49ca7ba3c-kube-api-access-xc8kl\") pod \"authorino-7bcfd8f9cd-5chsz\" (UID: \"c2e64835-1691-4481-9d8b-e2a49ca7ba3c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz"
Sep 16 13:41:25.655173 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.655165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95r2s\" (UniqueName: \"kubernetes.io/projected/13bffd4a-cb41-452b-a72d-0bc4d69b1f1d-kube-api-access-95r2s\") pod \"authorino-59c974bd48-lkkv2\" (UID: \"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d\") " pod="kuadrant-system/authorino-59c974bd48-lkkv2"
Sep 16 13:41:25.662541 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.662522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc8kl\" (UniqueName: \"kubernetes.io/projected/c2e64835-1691-4481-9d8b-e2a49ca7ba3c-kube-api-access-xc8kl\") pod \"authorino-7bcfd8f9cd-5chsz\" (UID: \"c2e64835-1691-4481-9d8b-e2a49ca7ba3c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz"
Sep 16 13:41:25.705451 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.705428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz"
Sep 16 13:41:25.756365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.756336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95r2s\" (UniqueName: \"kubernetes.io/projected/13bffd4a-cb41-452b-a72d-0bc4d69b1f1d-kube-api-access-95r2s\") pod \"authorino-59c974bd48-lkkv2\" (UID: \"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d\") " pod="kuadrant-system/authorino-59c974bd48-lkkv2"
Sep 16 13:41:25.765004 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.764947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95r2s\" (UniqueName: \"kubernetes.io/projected/13bffd4a-cb41-452b-a72d-0bc4d69b1f1d-kube-api-access-95r2s\") pod \"authorino-59c974bd48-lkkv2\" (UID: \"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d\") " pod="kuadrant-system/authorino-59c974bd48-lkkv2"
Sep 16 13:41:25.823016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.822995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lkkv2"
Sep 16 13:41:25.967943 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:25.967847    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5chsz"]
Sep 16 13:41:25.971721 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:41:25.971689    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2e64835_1691_4481_9d8b_e2a49ca7ba3c.slice/crio-c37a7ebb4b12b03d5d3676f3b96241631d0fcf80ab1187048c7a33237d8fec8e WatchSource:0}: Error finding container c37a7ebb4b12b03d5d3676f3b96241631d0fcf80ab1187048c7a33237d8fec8e: Status 404 returned error can't find the container with id c37a7ebb4b12b03d5d3676f3b96241631d0fcf80ab1187048c7a33237d8fec8e
Sep 16 13:41:26.087882 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:26.087858    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lkkv2"]
Sep 16 13:41:26.090467 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:41:26.090446    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13bffd4a_cb41_452b_a72d_0bc4d69b1f1d.slice/crio-12b008a4a70a7f54b1cf08f08654090a75076bec1161f629eedec877856f597f WatchSource:0}: Error finding container 12b008a4a70a7f54b1cf08f08654090a75076bec1161f629eedec877856f597f: Status 404 returned error can't find the container with id 12b008a4a70a7f54b1cf08f08654090a75076bec1161f629eedec877856f597f
Sep 16 13:41:26.631686 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:26.631624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz" event={"ID":"c2e64835-1691-4481-9d8b-e2a49ca7ba3c","Type":"ContainerStarted","Data":"c37a7ebb4b12b03d5d3676f3b96241631d0fcf80ab1187048c7a33237d8fec8e"}
Sep 16 13:41:26.633709 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:26.633662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lkkv2" event={"ID":"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d","Type":"ContainerStarted","Data":"12b008a4a70a7f54b1cf08f08654090a75076bec1161f629eedec877856f597f"}
Sep 16 13:41:28.643826 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:28.643790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz" event={"ID":"c2e64835-1691-4481-9d8b-e2a49ca7ba3c","Type":"ContainerStarted","Data":"b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189"}
Sep 16 13:41:28.645626 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:28.645600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lkkv2" event={"ID":"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d","Type":"ContainerStarted","Data":"b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85"}
Sep 16 13:41:28.656231 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:28.655714    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz" podStartSLOduration=1.055916227 podStartE2EDuration="3.655695459s" totalImagesPullingTime="2.599779232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:41:28.655413072 +0000 UTC m=+516.339071479" watchObservedRunningTime="2026-09-16 13:41:28.655695459 +0000 UTC m=+516.339353857"
Sep 16 13:41:28.668051 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:28.667998    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-lkkv2" podStartSLOduration=1.177238717 podStartE2EDuration="3.667965851s" totalImagesPullingTime="2.490727134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:41:28.667224933 +0000 UTC m=+516.350883328" watchObservedRunningTime="2026-09-16 13:41:28.667965851 +0000 UTC m=+516.351624251"
Sep 16 13:41:28.689401 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:28.689329    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5chsz"]
Sep 16 13:41:30.657098 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:30.656995    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz" podUID="c2e64835-1691-4481-9d8b-e2a49ca7ba3c" containerName="authorino" containerID="cri-o://b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189" gracePeriod=30
Sep 16 13:41:30.886819 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:30.886795    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz"
Sep 16 13:41:30.998582 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:30.998509    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xc8kl\" (UniqueName: \"kubernetes.io/projected/c2e64835-1691-4481-9d8b-e2a49ca7ba3c-kube-api-access-xc8kl\") pod \"c2e64835-1691-4481-9d8b-e2a49ca7ba3c\" (UID: \"c2e64835-1691-4481-9d8b-e2a49ca7ba3c\") "
Sep 16 13:41:31.000374 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.000348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e64835-1691-4481-9d8b-e2a49ca7ba3c-kube-api-access-xc8kl" (OuterVolumeSpecName: "kube-api-access-xc8kl") pod "c2e64835-1691-4481-9d8b-e2a49ca7ba3c" (UID: "c2e64835-1691-4481-9d8b-e2a49ca7ba3c"). InnerVolumeSpecName "kube-api-access-xc8kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:41:31.099551 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.099527    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc8kl\" (UniqueName: \"kubernetes.io/projected/c2e64835-1691-4481-9d8b-e2a49ca7ba3c-kube-api-access-xc8kl\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:41:31.657886 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.657857    2580 generic.go:356] "Generic (PLEG): container finished" podID="c2e64835-1691-4481-9d8b-e2a49ca7ba3c" containerID="b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189" exitCode=0
Sep 16 13:41:31.658278 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.657911    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz"
Sep 16 13:41:31.658278 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.657941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz" event={"ID":"c2e64835-1691-4481-9d8b-e2a49ca7ba3c","Type":"ContainerDied","Data":"b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189"}
Sep 16 13:41:31.658278 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.657986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5chsz" event={"ID":"c2e64835-1691-4481-9d8b-e2a49ca7ba3c","Type":"ContainerDied","Data":"c37a7ebb4b12b03d5d3676f3b96241631d0fcf80ab1187048c7a33237d8fec8e"}
Sep 16 13:41:31.658278 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.658007    2580 scope.go:117] "RemoveContainer" containerID="b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189"
Sep 16 13:41:31.666287 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.666256    2580 scope.go:117] "RemoveContainer" containerID="b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189"
Sep 16 13:41:31.666562 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:41:31.666541    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189\": container with ID starting with b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189 not found: ID does not exist" containerID="b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189"
Sep 16 13:41:31.666609 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.666571    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189"} err="failed to get container status \"b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189\": rpc error: code = NotFound desc = could not find container \"b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189\": container with ID starting with b24530b42bc1c51a2fcb3d1dec6e022b4bd04e5b7ccb00dbbb29d9a4030ba189 not found: ID does not exist"
Sep 16 13:41:31.676350 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.676329    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5chsz"]
Sep 16 13:41:31.679925 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:31.679888    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5chsz"]
Sep 16 13:41:32.801048 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:32.801018    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e64835-1691-4481-9d8b-e2a49ca7ba3c" path="/var/lib/kubelet/pods/c2e64835-1691-4481-9d8b-e2a49ca7ba3c/volumes"
Sep 16 13:41:56.245916 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.245864    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-sj4wt"]
Sep 16 13:41:56.246468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.246241    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e64835-1691-4481-9d8b-e2a49ca7ba3c" containerName="authorino"
Sep 16 13:41:56.246468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.246260    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e64835-1691-4481-9d8b-e2a49ca7ba3c" containerName="authorino"
Sep 16 13:41:56.246468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.246321    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e64835-1691-4481-9d8b-e2a49ca7ba3c" containerName="authorino"
Sep 16 13:41:56.250318 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.250298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sj4wt"
Sep 16 13:41:56.253493 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.253469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-sj4wt"]
Sep 16 13:41:56.260585 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.260566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dfjk\" (UniqueName: \"kubernetes.io/projected/79a45ca3-3630-4fe8-bca3-b1f6f23ad5df-kube-api-access-4dfjk\") pod \"authorino-557f876f69-sj4wt\" (UID: \"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df\") " pod="kuadrant-system/authorino-557f876f69-sj4wt"
Sep 16 13:41:56.361011 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.360987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dfjk\" (UniqueName: \"kubernetes.io/projected/79a45ca3-3630-4fe8-bca3-b1f6f23ad5df-kube-api-access-4dfjk\") pod \"authorino-557f876f69-sj4wt\" (UID: \"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df\") " pod="kuadrant-system/authorino-557f876f69-sj4wt"
Sep 16 13:41:56.368521 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.368494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dfjk\" (UniqueName: \"kubernetes.io/projected/79a45ca3-3630-4fe8-bca3-b1f6f23ad5df-kube-api-access-4dfjk\") pod \"authorino-557f876f69-sj4wt\" (UID: \"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df\") " pod="kuadrant-system/authorino-557f876f69-sj4wt"
Sep 16 13:41:56.507441 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.507388    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-sj4wt"]
Sep 16 13:41:56.507587 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.507574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sj4wt"
Sep 16 13:41:56.534075 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.534051    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-695d5cd648-dl2tp"]
Sep 16 13:41:56.537267 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.537249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695d5cd648-dl2tp"
Sep 16 13:41:56.544164 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.544114    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-695d5cd648-dl2tp"]
Sep 16 13:41:56.563419 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.563389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q8xm\" (UniqueName: \"kubernetes.io/projected/8d284001-3513-404e-9743-0013469442b5-kube-api-access-7q8xm\") pod \"authorino-695d5cd648-dl2tp\" (UID: \"8d284001-3513-404e-9743-0013469442b5\") " pod="kuadrant-system/authorino-695d5cd648-dl2tp"
Sep 16 13:41:56.664428 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.664400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q8xm\" (UniqueName: \"kubernetes.io/projected/8d284001-3513-404e-9743-0013469442b5-kube-api-access-7q8xm\") pod \"authorino-695d5cd648-dl2tp\" (UID: \"8d284001-3513-404e-9743-0013469442b5\") " pod="kuadrant-system/authorino-695d5cd648-dl2tp"
Sep 16 13:41:56.671909 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.671878    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q8xm\" (UniqueName: \"kubernetes.io/projected/8d284001-3513-404e-9743-0013469442b5-kube-api-access-7q8xm\") pod \"authorino-695d5cd648-dl2tp\" (UID: \"8d284001-3513-404e-9743-0013469442b5\") " pod="kuadrant-system/authorino-695d5cd648-dl2tp"
Sep 16 13:41:56.775412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.775353    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-sj4wt"]
Sep 16 13:41:56.778535 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:41:56.778513    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79a45ca3_3630_4fe8_bca3_b1f6f23ad5df.slice/crio-89aff4472462d35504aaaff76bf2e5b573f56ec8ef0eb7f42700388235e27697 WatchSource:0}: Error finding container 89aff4472462d35504aaaff76bf2e5b573f56ec8ef0eb7f42700388235e27697: Status 404 returned error can't find the container with id 89aff4472462d35504aaaff76bf2e5b573f56ec8ef0eb7f42700388235e27697
Sep 16 13:41:56.847850 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:56.847828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695d5cd648-dl2tp"
Sep 16 13:41:57.113551 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.113486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-695d5cd648-dl2tp"]
Sep 16 13:41:57.116752 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:41:57.116728    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d284001_3513_404e_9743_0013469442b5.slice/crio-88fa540d13087bd5d14b3032b90a5155b6e7e1642edf0c9238a19ddc3141fc5a WatchSource:0}: Error finding container 88fa540d13087bd5d14b3032b90a5155b6e7e1642edf0c9238a19ddc3141fc5a: Status 404 returned error can't find the container with id 88fa540d13087bd5d14b3032b90a5155b6e7e1642edf0c9238a19ddc3141fc5a
Sep 16 13:41:57.758626 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.758590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695d5cd648-dl2tp" event={"ID":"8d284001-3513-404e-9743-0013469442b5","Type":"ContainerStarted","Data":"e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0"}
Sep 16 13:41:57.759067 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.758641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695d5cd648-dl2tp" event={"ID":"8d284001-3513-404e-9743-0013469442b5","Type":"ContainerStarted","Data":"88fa540d13087bd5d14b3032b90a5155b6e7e1642edf0c9238a19ddc3141fc5a"}
Sep 16 13:41:57.760155 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.760127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sj4wt" event={"ID":"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df","Type":"ContainerStarted","Data":"552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f"}
Sep 16 13:41:57.760264 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.760165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sj4wt" event={"ID":"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df","Type":"ContainerStarted","Data":"89aff4472462d35504aaaff76bf2e5b573f56ec8ef0eb7f42700388235e27697"}
Sep 16 13:41:57.760264 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.760213    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-sj4wt" podUID="79a45ca3-3630-4fe8-bca3-b1f6f23ad5df" containerName="authorino" containerID="cri-o://552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f" gracePeriod=30
Sep 16 13:41:57.773350 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.773309    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-695d5cd648-dl2tp" podStartSLOduration=1.379062238 podStartE2EDuration="1.773294673s" totalImagesPullingTime="394.232435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:41:57.771534846 +0000 UTC m=+545.455193241" watchObservedRunningTime="2026-09-16 13:41:57.773294673 +0000 UTC m=+545.456953067"
Sep 16 13:41:57.785341 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.785289    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-sj4wt" podStartSLOduration=1.257703512 podStartE2EDuration="1.785269953s" totalImagesPullingTime="527.566441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:41:57.783987026 +0000 UTC m=+545.467645420" watchObservedRunningTime="2026-09-16 13:41:57.785269953 +0000 UTC m=+545.468928347"
Sep 16 13:41:57.798461 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.798438    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lkkv2"]
Sep 16 13:41:57.798588 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.798572    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-lkkv2" podUID="13bffd4a-cb41-452b-a72d-0bc4d69b1f1d" containerName="authorino" containerID="cri-o://b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85" gracePeriod=30
Sep 16 13:41:57.880325 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.880293    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-789d6b7b54-xj7mm"]
Sep 16 13:41:57.883138 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.883122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:57.884874 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.884853    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 13:41:57.889573 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.889551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-789d6b7b54-xj7mm"]
Sep 16 13:41:57.975469 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.975440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz528\" (UniqueName: \"kubernetes.io/projected/1cae7f23-debd-4c2d-b294-1f030764aefd-kube-api-access-qz528\") pod \"authorino-789d6b7b54-xj7mm\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") " pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:57.975601 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:57.975502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1cae7f23-debd-4c2d-b294-1f030764aefd-tls-cert\") pod \"authorino-789d6b7b54-xj7mm\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") " pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:58.043548 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.043426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sj4wt"
Sep 16 13:41:58.050069 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.050050    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lkkv2"
Sep 16 13:41:58.076578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.076552    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95r2s\" (UniqueName: \"kubernetes.io/projected/13bffd4a-cb41-452b-a72d-0bc4d69b1f1d-kube-api-access-95r2s\") pod \"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d\" (UID: \"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d\") "
Sep 16 13:41:58.076719 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.076611    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dfjk\" (UniqueName: \"kubernetes.io/projected/79a45ca3-3630-4fe8-bca3-b1f6f23ad5df-kube-api-access-4dfjk\") pod \"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df\" (UID: \"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df\") "
Sep 16 13:41:58.076719 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.076679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz528\" (UniqueName: \"kubernetes.io/projected/1cae7f23-debd-4c2d-b294-1f030764aefd-kube-api-access-qz528\") pod \"authorino-789d6b7b54-xj7mm\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") " pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:58.076843 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.076720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1cae7f23-debd-4c2d-b294-1f030764aefd-tls-cert\") pod \"authorino-789d6b7b54-xj7mm\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") " pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:58.078812 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.078787    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13bffd4a-cb41-452b-a72d-0bc4d69b1f1d-kube-api-access-95r2s" (OuterVolumeSpecName: "kube-api-access-95r2s") pod "13bffd4a-cb41-452b-a72d-0bc4d69b1f1d" (UID: "13bffd4a-cb41-452b-a72d-0bc4d69b1f1d"). InnerVolumeSpecName "kube-api-access-95r2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:41:58.078941 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.078840    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a45ca3-3630-4fe8-bca3-b1f6f23ad5df-kube-api-access-4dfjk" (OuterVolumeSpecName: "kube-api-access-4dfjk") pod "79a45ca3-3630-4fe8-bca3-b1f6f23ad5df" (UID: "79a45ca3-3630-4fe8-bca3-b1f6f23ad5df"). InnerVolumeSpecName "kube-api-access-4dfjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:41:58.079277 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.079259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1cae7f23-debd-4c2d-b294-1f030764aefd-tls-cert\") pod \"authorino-789d6b7b54-xj7mm\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") " pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:58.083470 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.083452    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz528\" (UniqueName: \"kubernetes.io/projected/1cae7f23-debd-4c2d-b294-1f030764aefd-kube-api-access-qz528\") pod \"authorino-789d6b7b54-xj7mm\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") " pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:58.178014 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.177989    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dfjk\" (UniqueName: \"kubernetes.io/projected/79a45ca3-3630-4fe8-bca3-b1f6f23ad5df-kube-api-access-4dfjk\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:41:58.178014 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.178014    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95r2s\" (UniqueName: \"kubernetes.io/projected/13bffd4a-cb41-452b-a72d-0bc4d69b1f1d-kube-api-access-95r2s\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:41:58.191984 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.191963    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:41:58.458612 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.458588    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-789d6b7b54-xj7mm"]
Sep 16 13:41:58.461628 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:41:58.461607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cae7f23_debd_4c2d_b294_1f030764aefd.slice/crio-7264c5798006512d7edf745d298a999ee576f51669605d766e4dae8e09d71ae5 WatchSource:0}: Error finding container 7264c5798006512d7edf745d298a999ee576f51669605d766e4dae8e09d71ae5: Status 404 returned error can't find the container with id 7264c5798006512d7edf745d298a999ee576f51669605d766e4dae8e09d71ae5
Sep 16 13:41:58.769820 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.769788    2580 generic.go:356] "Generic (PLEG): container finished" podID="79a45ca3-3630-4fe8-bca3-b1f6f23ad5df" containerID="552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f" exitCode=0
Sep 16 13:41:58.770267 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.769846    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sj4wt"
Sep 16 13:41:58.770267 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.769872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sj4wt" event={"ID":"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df","Type":"ContainerDied","Data":"552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f"}
Sep 16 13:41:58.770267 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.769927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sj4wt" event={"ID":"79a45ca3-3630-4fe8-bca3-b1f6f23ad5df","Type":"ContainerDied","Data":"89aff4472462d35504aaaff76bf2e5b573f56ec8ef0eb7f42700388235e27697"}
Sep 16 13:41:58.771376 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.771351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789d6b7b54-xj7mm" event={"ID":"1cae7f23-debd-4c2d-b294-1f030764aefd","Type":"ContainerStarted","Data":"7264c5798006512d7edf745d298a999ee576f51669605d766e4dae8e09d71ae5"}
Sep 16 13:41:58.771850 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.769956    2580 scope.go:117] "RemoveContainer" containerID="552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f"
Sep 16 13:41:58.772604 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.772581    2580 generic.go:356] "Generic (PLEG): container finished" podID="13bffd4a-cb41-452b-a72d-0bc4d69b1f1d" containerID="b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85" exitCode=0
Sep 16 13:41:58.772684 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.772642    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lkkv2"
Sep 16 13:41:58.772684 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.772646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lkkv2" event={"ID":"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d","Type":"ContainerDied","Data":"b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85"}
Sep 16 13:41:58.772784 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.772685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lkkv2" event={"ID":"13bffd4a-cb41-452b-a72d-0bc4d69b1f1d","Type":"ContainerDied","Data":"12b008a4a70a7f54b1cf08f08654090a75076bec1161f629eedec877856f597f"}
Sep 16 13:41:58.779755 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.779738    2580 scope.go:117] "RemoveContainer" containerID="552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f"
Sep 16 13:41:58.780014 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:41:58.779985    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f\": container with ID starting with 552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f not found: ID does not exist" containerID="552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f"
Sep 16 13:41:58.780096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.780011    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f"} err="failed to get container status \"552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f\": rpc error: code = NotFound desc = could not find container \"552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f\": container with ID starting with 552a8a85d1a3aec0d9f61c7295468c99fa9d4511cb5dd5c4fe1266a792c8744f not found: ID does not exist"
Sep 16 13:41:58.780096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.780041    2580 scope.go:117] "RemoveContainer" containerID="b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85"
Sep 16 13:41:58.787182 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.787160    2580 scope.go:117] "RemoveContainer" containerID="b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85"
Sep 16 13:41:58.787444 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:41:58.787427    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85\": container with ID starting with b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85 not found: ID does not exist" containerID="b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85"
Sep 16 13:41:58.787499 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.787450    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85"} err="failed to get container status \"b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85\": rpc error: code = NotFound desc = could not find container \"b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85\": container with ID starting with b67a030a5fc4a85dac605fe318d59112a552bcf898a5d3da9a53d84893583a85 not found: ID does not exist"
Sep 16 13:41:58.792052 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.792032    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-sj4wt"]
Sep 16 13:41:58.795601 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.795582    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-sj4wt"]
Sep 16 13:41:58.801492 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.801473    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79a45ca3-3630-4fe8-bca3-b1f6f23ad5df" path="/var/lib/kubelet/pods/79a45ca3-3630-4fe8-bca3-b1f6f23ad5df/volumes"
Sep 16 13:41:58.804926 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.804891    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lkkv2"]
Sep 16 13:41:58.808403 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:58.808385    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lkkv2"]
Sep 16 13:41:59.778808 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:59.778775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789d6b7b54-xj7mm" event={"ID":"1cae7f23-debd-4c2d-b294-1f030764aefd","Type":"ContainerStarted","Data":"6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23"}
Sep 16 13:41:59.791845 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:59.791736    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-789d6b7b54-xj7mm" podStartSLOduration=2.240960368 podStartE2EDuration="2.791719101s" totalImagesPullingTime="550.758733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:41:59.791643696 +0000 UTC m=+547.475302090" watchObservedRunningTime="2026-09-16 13:41:59.791719101 +0000 UTC m=+547.475377495"
Sep 16 13:41:59.815107 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:59.815073    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-695d5cd648-dl2tp"]
Sep 16 13:41:59.815231 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:41:59.815210    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-695d5cd648-dl2tp" podUID="8d284001-3513-404e-9743-0013469442b5" containerName="authorino" containerID="cri-o://e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0" gracePeriod=30
Sep 16 13:42:00.049443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.049423    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695d5cd648-dl2tp"
Sep 16 13:42:00.092524 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.092499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q8xm\" (UniqueName: \"kubernetes.io/projected/8d284001-3513-404e-9743-0013469442b5-kube-api-access-7q8xm\") pod \"8d284001-3513-404e-9743-0013469442b5\" (UID: \"8d284001-3513-404e-9743-0013469442b5\") "
Sep 16 13:42:00.094398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.094365    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d284001-3513-404e-9743-0013469442b5-kube-api-access-7q8xm" (OuterVolumeSpecName: "kube-api-access-7q8xm") pod "8d284001-3513-404e-9743-0013469442b5" (UID: "8d284001-3513-404e-9743-0013469442b5"). InnerVolumeSpecName "kube-api-access-7q8xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:42:00.192863 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.192836    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q8xm\" (UniqueName: \"kubernetes.io/projected/8d284001-3513-404e-9743-0013469442b5-kube-api-access-7q8xm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:42:00.784678 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.784646    2580 generic.go:356] "Generic (PLEG): container finished" podID="8d284001-3513-404e-9743-0013469442b5" containerID="e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0" exitCode=0
Sep 16 13:42:00.785115 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.784691    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695d5cd648-dl2tp"
Sep 16 13:42:00.785115 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.784735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695d5cd648-dl2tp" event={"ID":"8d284001-3513-404e-9743-0013469442b5","Type":"ContainerDied","Data":"e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0"}
Sep 16 13:42:00.785115 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.784775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695d5cd648-dl2tp" event={"ID":"8d284001-3513-404e-9743-0013469442b5","Type":"ContainerDied","Data":"88fa540d13087bd5d14b3032b90a5155b6e7e1642edf0c9238a19ddc3141fc5a"}
Sep 16 13:42:00.785115 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.784796    2580 scope.go:117] "RemoveContainer" containerID="e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0"
Sep 16 13:42:00.792888 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.792874    2580 scope.go:117] "RemoveContainer" containerID="e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0"
Sep 16 13:42:00.793149 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:42:00.793127    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0\": container with ID starting with e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0 not found: ID does not exist" containerID="e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0"
Sep 16 13:42:00.793207 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.793157    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0"} err="failed to get container status \"e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0\": rpc error: code = NotFound desc = could not find container \"e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0\": container with ID starting with e908e124118921723151774dd7082697e845b9384354005531cc8afb8bdfd6c0 not found: ID does not exist"
Sep 16 13:42:00.801406 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.801381    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13bffd4a-cb41-452b-a72d-0bc4d69b1f1d" path="/var/lib/kubelet/pods/13bffd4a-cb41-452b-a72d-0bc4d69b1f1d/volumes"
Sep 16 13:42:00.802147 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.802128    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-695d5cd648-dl2tp"]
Sep 16 13:42:00.805327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:00.805298    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-695d5cd648-dl2tp"]
Sep 16 13:42:02.760719 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.760689    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6559cb7db7-xhkzp"]
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.760966    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79a45ca3-3630-4fe8-bca3-b1f6f23ad5df" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.760980    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="79a45ca3-3630-4fe8-bca3-b1f6f23ad5df" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.760991    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d284001-3513-404e-9743-0013469442b5" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.760997    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d284001-3513-404e-9743-0013469442b5" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.761012    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13bffd4a-cb41-452b-a72d-0bc4d69b1f1d" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.761017    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="13bffd4a-cb41-452b-a72d-0bc4d69b1f1d" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.761075    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="79a45ca3-3630-4fe8-bca3-b1f6f23ad5df" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.761083    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d284001-3513-404e-9743-0013469442b5" containerName="authorino"
Sep 16 13:42:02.761093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.761088    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="13bffd4a-cb41-452b-a72d-0bc4d69b1f1d" containerName="authorino"
Sep 16 13:42:02.765414 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.765395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.767516 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.767494    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m6pl2\""
Sep 16 13:42:02.767646 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.767496    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 13:42:02.767646 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.767565    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 13:42:02.767986 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.767972    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 13:42:02.772415 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.771989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6559cb7db7-xhkzp"]
Sep 16 13:42:02.801835 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.801806    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d284001-3513-404e-9743-0013469442b5" path="/var/lib/kubelet/pods/8d284001-3513-404e-9743-0013469442b5/volumes"
Sep 16 13:42:02.811480 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.811459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-metrics-certs\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.811550 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.811499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.811550 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.811530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grvxx\" (UniqueName: \"kubernetes.io/projected/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-kube-api-access-grvxx\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.912139 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.912112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-metrics-certs\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.912241 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.912151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.912241 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.912192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grvxx\" (UniqueName: \"kubernetes.io/projected/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-kube-api-access-grvxx\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.912332 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:42:02.912266    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 16 13:42:02.912368 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:42:02.912335    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert podName:a7bb6c6e-9610-49f5-a039-d1b6cb8033b1 nodeName:}" failed. No retries permitted until 2026-09-16 13:42:03.412319308 +0000 UTC m=+551.095977680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert") pod "maas-controller-6559cb7db7-xhkzp" (UID: "a7bb6c6e-9610-49f5-a039-d1b6cb8033b1") : secret "maas-controller-webhook-cert" not found
Sep 16 13:42:02.914500 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.914479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-metrics-certs\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.920922 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.920884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grvxx\" (UniqueName: \"kubernetes.io/projected/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-kube-api-access-grvxx\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:02.937152 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.937134    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f96f875d-qwvrj"]
Sep 16 13:42:02.939526 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.939512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:02.948189 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:02.948121    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f96f875d-qwvrj"]
Sep 16 13:42:03.013517 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.013450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-kube-api-access-cqhtn\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.013517 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.013486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-metrics-certs\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.013661 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.013577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-webhook-cert\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.114267 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.114244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-kube-api-access-cqhtn\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.114383 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.114271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-metrics-certs\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.114383 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.114314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-webhook-cert\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.116568 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.116540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-metrics-certs\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.116714 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.116695    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-webhook-cert\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.121143 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.121127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-kube-api-access-cqhtn\") pod \"maas-controller-7f96f875d-qwvrj\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") " pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.249708 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.249678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:03.415815 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.415784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:03.417943 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.417925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert\") pod \"maas-controller-6559cb7db7-xhkzp\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") " pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:03.521557 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.521529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f96f875d-qwvrj"]
Sep 16 13:42:03.524603 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:42:03.524577    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23acd4b0_a3d3_4253_9a49_583af9bb8ddb.slice/crio-2407987c16b86491ce53799acc3d24f9362fc694f7217aec1e2b2c5d9185454e WatchSource:0}: Error finding container 2407987c16b86491ce53799acc3d24f9362fc694f7217aec1e2b2c5d9185454e: Status 404 returned error can't find the container with id 2407987c16b86491ce53799acc3d24f9362fc694f7217aec1e2b2c5d9185454e
Sep 16 13:42:03.675491 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.675469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:03.803190 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.803161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-qwvrj" event={"ID":"23acd4b0-a3d3-4253-9a49-583af9bb8ddb","Type":"ContainerStarted","Data":"2407987c16b86491ce53799acc3d24f9362fc694f7217aec1e2b2c5d9185454e"}
Sep 16 13:42:03.943003 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:03.942836    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6559cb7db7-xhkzp"]
Sep 16 13:42:03.945227 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:42:03.945203    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bb6c6e_9610_49f5_a039_d1b6cb8033b1.slice/crio-7902c2937a44a9350477355182ebb501bd0f4d45488c6c6fe12cd2acf31a6fdc WatchSource:0}: Error finding container 7902c2937a44a9350477355182ebb501bd0f4d45488c6c6fe12cd2acf31a6fdc: Status 404 returned error can't find the container with id 7902c2937a44a9350477355182ebb501bd0f4d45488c6c6fe12cd2acf31a6fdc
Sep 16 13:42:04.810172 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:04.810134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6559cb7db7-xhkzp" event={"ID":"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1","Type":"ContainerStarted","Data":"7902c2937a44a9350477355182ebb501bd0f4d45488c6c6fe12cd2acf31a6fdc"}
Sep 16 13:42:06.820675 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:06.820588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-qwvrj" event={"ID":"23acd4b0-a3d3-4253-9a49-583af9bb8ddb","Type":"ContainerStarted","Data":"4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33"}
Sep 16 13:42:06.821140 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:06.820758    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:06.822848 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:06.822823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6559cb7db7-xhkzp" event={"ID":"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1","Type":"ContainerStarted","Data":"9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49"}
Sep 16 13:42:06.823076 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:06.823051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:06.840665 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:06.839298    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f96f875d-qwvrj" podStartSLOduration=1.859623926 podStartE2EDuration="4.839280256s" totalImagesPullingTime="2.97965633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:42:06.838513883 +0000 UTC m=+554.522172278" watchObservedRunningTime="2026-09-16 13:42:06.839280256 +0000 UTC m=+554.522938652"
Sep 16 13:42:06.855486 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:06.855445    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6559cb7db7-xhkzp" podStartSLOduration=2.305372588 podStartE2EDuration="4.855431477s" totalImagesPullingTime="2.550058889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:42:06.853676243 +0000 UTC m=+554.537334638" watchObservedRunningTime="2026-09-16 13:42:06.855431477 +0000 UTC m=+554.539089872"
Sep 16 13:42:17.828550 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:17.828519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:17.828982 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:17.828600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 13:42:17.882894 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:17.882860    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6559cb7db7-xhkzp"]
Sep 16 13:42:17.883138 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:17.883112    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6559cb7db7-xhkzp" podUID="a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" containerName="manager" containerID="cri-o://9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49" gracePeriod=30
Sep 16 13:42:18.123283 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.123261    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:18.225671 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.225645    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grvxx\" (UniqueName: \"kubernetes.io/projected/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-kube-api-access-grvxx\") pod \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") "
Sep 16 13:42:18.225810 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.225692    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-metrics-certs\") pod \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") "
Sep 16 13:42:18.225810 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.225712    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert\") pod \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\" (UID: \"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1\") "
Sep 16 13:42:18.227859 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.227830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" (UID: "a7bb6c6e-9610-49f5-a039-d1b6cb8033b1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:42:18.227967 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.227885    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-kube-api-access-grvxx" (OuterVolumeSpecName: "kube-api-access-grvxx") pod "a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" (UID: "a7bb6c6e-9610-49f5-a039-d1b6cb8033b1"). InnerVolumeSpecName "kube-api-access-grvxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:42:18.227967 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.227884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" (UID: "a7bb6c6e-9610-49f5-a039-d1b6cb8033b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:42:18.326318 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.326289    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:42:18.326318 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.326311    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-webhook-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:42:18.326318 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.326320    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grvxx\" (UniqueName: \"kubernetes.io/projected/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1-kube-api-access-grvxx\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:42:18.870450 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.870424    2580 generic.go:356] "Generic (PLEG): container finished" podID="a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" containerID="9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49" exitCode=0
Sep 16 13:42:18.870835 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.870469    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6559cb7db7-xhkzp"
Sep 16 13:42:18.870835 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.870506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6559cb7db7-xhkzp" event={"ID":"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1","Type":"ContainerDied","Data":"9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49"}
Sep 16 13:42:18.870835 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.870558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6559cb7db7-xhkzp" event={"ID":"a7bb6c6e-9610-49f5-a039-d1b6cb8033b1","Type":"ContainerDied","Data":"7902c2937a44a9350477355182ebb501bd0f4d45488c6c6fe12cd2acf31a6fdc"}
Sep 16 13:42:18.870835 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.870581    2580 scope.go:117] "RemoveContainer" containerID="9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49"
Sep 16 13:42:18.878652 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.878627    2580 scope.go:117] "RemoveContainer" containerID="9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49"
Sep 16 13:42:18.878934 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:42:18.878914    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49\": container with ID starting with 9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49 not found: ID does not exist" containerID="9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49"
Sep 16 13:42:18.878987 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.878943    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49"} err="failed to get container status \"9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49\": rpc error: code = NotFound desc = could not find container \"9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49\": container with ID starting with 9df394ca6b723e85294cfad298e32856323554c4ec924fca9cbfe8145f366d49 not found: ID does not exist"
Sep 16 13:42:18.898529 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.898505    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6559cb7db7-xhkzp"]
Sep 16 13:42:18.906584 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:18.906567    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6559cb7db7-xhkzp"]
Sep 16 13:42:20.801198 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:20.801167    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" path="/var/lib/kubelet/pods/a7bb6c6e-9610-49f5-a039-d1b6cb8033b1/volumes"
Sep 16 13:42:41.269591 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.269555    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"]
Sep 16 13:42:41.270126 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.269936    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" containerName="manager"
Sep 16 13:42:41.270126 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.269953    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" containerName="manager"
Sep 16 13:42:41.270126 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.270011    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7bb6c6e-9610-49f5-a039-d1b6cb8033b1" containerName="manager"
Sep 16 13:42:41.274235 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.274205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.276244 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.276216    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 13:42:41.283802 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.283778    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"]
Sep 16 13:42:41.375177 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.375131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b109d40-55e2-4176-81f4-28f9197df298-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-pzqqj\" (UID: \"3b109d40-55e2-4176-81f4-28f9197df298\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.375356 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.375190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkn5d\" (UniqueName: \"kubernetes.io/projected/3b109d40-55e2-4176-81f4-28f9197df298-kube-api-access-nkn5d\") pod \"payload-pre-processing-59fbc47bf8-pzqqj\" (UID: \"3b109d40-55e2-4176-81f4-28f9197df298\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.476326 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.476288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b109d40-55e2-4176-81f4-28f9197df298-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-pzqqj\" (UID: \"3b109d40-55e2-4176-81f4-28f9197df298\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.476326 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.476329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkn5d\" (UniqueName: \"kubernetes.io/projected/3b109d40-55e2-4176-81f4-28f9197df298-kube-api-access-nkn5d\") pod \"payload-pre-processing-59fbc47bf8-pzqqj\" (UID: \"3b109d40-55e2-4176-81f4-28f9197df298\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.477349 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.477307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b109d40-55e2-4176-81f4-28f9197df298-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-pzqqj\" (UID: \"3b109d40-55e2-4176-81f4-28f9197df298\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.484985 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.484958    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkn5d\" (UniqueName: \"kubernetes.io/projected/3b109d40-55e2-4176-81f4-28f9197df298-kube-api-access-nkn5d\") pod \"payload-pre-processing-59fbc47bf8-pzqqj\" (UID: \"3b109d40-55e2-4176-81f4-28f9197df298\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.588719 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.588633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:41.859501 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.859436    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"]
Sep 16 13:42:41.862713 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:42:41.862687    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b109d40_55e2_4176_81f4_28f9197df298.slice/crio-4fb57266715b3e81ab7b380d76e6b49ea094646b9e54a730efd283a59998013e WatchSource:0}: Error finding container 4fb57266715b3e81ab7b380d76e6b49ea094646b9e54a730efd283a59998013e: Status 404 returned error can't find the container with id 4fb57266715b3e81ab7b380d76e6b49ea094646b9e54a730efd283a59998013e
Sep 16 13:42:41.957168 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:41.957139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj" event={"ID":"3b109d40-55e2-4176-81f4-28f9197df298","Type":"ContainerStarted","Data":"4fb57266715b3e81ab7b380d76e6b49ea094646b9e54a730efd283a59998013e"}
Sep 16 13:42:44.971260 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:44.971227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj" event={"ID":"3b109d40-55e2-4176-81f4-28f9197df298","Type":"ContainerStarted","Data":"834743f1b660e065b95e15965113a683385b3c0b7c738244c83e3376562a51ec"}
Sep 16 13:42:44.971607 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:44.971298    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:42:44.985959 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:44.985917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj" podStartSLOduration=1.6765018569999999 podStartE2EDuration="3.985888151s" totalImagesPullingTime="2.309386294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:42:44.984548531 +0000 UTC m=+592.668206929" watchObservedRunningTime="2026-09-16 13:42:44.985888151 +0000 UTC m=+592.669546545"
Sep 16 13:42:52.690917 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:52.690878    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:42:52.700670 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:52.700648    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:42:55.977455 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:42:55.977423    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-pzqqj"
Sep 16 13:43:00.247798 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.247765    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"]
Sep 16 13:43:00.252061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.252033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.254088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.254070    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xh5w6\""
Sep 16 13:43:00.254184 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.254125    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 13:43:00.254828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.254814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 13:43:00.254938 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.254818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 16 13:43:00.261202 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.261182    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"]
Sep 16 13:43:00.404459 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.404427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.404459 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.404459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.404634 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.404484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.404634 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.404502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.404634 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.404565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkbtd\" (UniqueName: \"kubernetes.io/projected/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-kube-api-access-qkbtd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.404634 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.404623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.505856 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.505767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.505856 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.505822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.506084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.505939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.506084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.505983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.506084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.506002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.506084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.506035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkbtd\" (UniqueName: \"kubernetes.io/projected/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-kube-api-access-qkbtd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.506275 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.506128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.506333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.506301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.506424 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.506403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.508035 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.508015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.508392 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.508371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.513501 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.513474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkbtd\" (UniqueName: \"kubernetes.io/projected/ecc22a7d-6d9f-493e-9aa1-9d8edcb98690-kube-api-access-qkbtd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4\" (UID: \"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.561914 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.561874    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:00.834512 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:00.834424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"]
Sep 16 13:43:00.841840 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:43:00.841801    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc22a7d_6d9f_493e_9aa1_9d8edcb98690.slice/crio-2537923b5917f088fad9d9d24c834d47d0b2fbf586a71e4094caf6f265a1c4fa WatchSource:0}: Error finding container 2537923b5917f088fad9d9d24c834d47d0b2fbf586a71e4094caf6f265a1c4fa: Status 404 returned error can't find the container with id 2537923b5917f088fad9d9d24c834d47d0b2fbf586a71e4094caf6f265a1c4fa
Sep 16 13:43:01.038682 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:01.038627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4" event={"ID":"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690","Type":"ContainerStarted","Data":"2537923b5917f088fad9d9d24c834d47d0b2fbf586a71e4094caf6f265a1c4fa"}
Sep 16 13:43:02.044316 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:02.044276    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4" event={"ID":"ecc22a7d-6d9f-493e-9aa1-9d8edcb98690","Type":"ContainerStarted","Data":"a30dcace5098eb56bd55f2f246a2ffe49e40c5188439f739ee07d1a0e65262a3"}
Sep 16 13:43:02.059847 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:02.059792    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4" podStartSLOduration=0.974541904 podStartE2EDuration="2.059777391s" totalImagesPullingTime="1.085235487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:43:02.059196839 +0000 UTC m=+609.742855235" watchObservedRunningTime="2026-09-16 13:43:02.059777391 +0000 UTC m=+609.743435785"
Sep 16 13:43:10.562452 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:10.562418    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:10.565025 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:10.562483    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:10.575085 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:10.575064    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:11.088203 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:11.088174    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4"
Sep 16 13:43:43.842571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.842534    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"]
Sep 16 13:43:43.848081 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.848062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:43.850101 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.850074    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 16 13:43:43.857394 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.857368    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"]
Sep 16 13:43:43.914957 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.914936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnqgm\" (UniqueName: \"kubernetes.io/projected/cd6e05bd-e925-4574-8f68-7ee265162c55-kube-api-access-dnqgm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:43.915064 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.914984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6e05bd-e925-4574-8f68-7ee265162c55-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:43.915118 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.915069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:43.915118 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.915097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:43.915193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.915123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:43.915193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:43.915142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.015819 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.015794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6e05bd-e925-4574-8f68-7ee265162c55-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.015946 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.015836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.015946 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.015856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.016063 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.015969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.016063 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.016007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.016063 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.016048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnqgm\" (UniqueName: \"kubernetes.io/projected/cd6e05bd-e925-4574-8f68-7ee265162c55-kube-api-access-dnqgm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.016243 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.016222    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.016292 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.016268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.016393 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.016373    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.018408 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.018385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6e05bd-e925-4574-8f68-7ee265162c55-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.018555 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.018535    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6e05bd-e925-4574-8f68-7ee265162c55-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.023510 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.023481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnqgm\" (UniqueName: \"kubernetes.io/projected/cd6e05bd-e925-4574-8f68-7ee265162c55-kube-api-access-dnqgm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f\" (UID: \"cd6e05bd-e925-4574-8f68-7ee265162c55\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.158543 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.158516    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:44.435078 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.434936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"]
Sep 16 13:43:44.437654 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:43:44.437617    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd6e05bd_e925_4574_8f68_7ee265162c55.slice/crio-d1a7c92f2d67ec4fc8a1048c366de847cb8e9e3c145939dc87fc1ede2aa1c872 WatchSource:0}: Error finding container d1a7c92f2d67ec4fc8a1048c366de847cb8e9e3c145939dc87fc1ede2aa1c872: Status 404 returned error can't find the container with id d1a7c92f2d67ec4fc8a1048c366de847cb8e9e3c145939dc87fc1ede2aa1c872
Sep 16 13:43:44.438964 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:44.438946    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 13:43:45.213715 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:45.213682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f" event={"ID":"cd6e05bd-e925-4574-8f68-7ee265162c55","Type":"ContainerStarted","Data":"630ffc1bc3f0a93b119975c6560ce27d3f64fa8de3f7b9f3eee1d5c31fc69fd4"}
Sep 16 13:43:45.214081 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:45.213731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f" event={"ID":"cd6e05bd-e925-4574-8f68-7ee265162c55","Type":"ContainerStarted","Data":"d1a7c92f2d67ec4fc8a1048c366de847cb8e9e3c145939dc87fc1ede2aa1c872"}
Sep 16 13:43:45.233176 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:45.233128    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f" podStartSLOduration=2.035852269 podStartE2EDuration="2.23311147s" totalImagesPullingTime="197.259201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:43:45.231891756 +0000 UTC m=+652.915550155" watchObservedRunningTime="2026-09-16 13:43:45.23311147 +0000 UTC m=+652.916769861"
Sep 16 13:43:54.159591 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:54.159556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:54.159993 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:54.159616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:54.171914 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:54.171881    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:54.260455 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:54.260425    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f"
Sep 16 13:43:59.331849 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.331809    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"]
Sep 16 13:43:59.335052 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.335032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.337030 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.337010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 16 13:43:59.349488 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.349465    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"]
Sep 16 13:43:59.425187 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.425160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.425295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.425194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.425295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.425222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c48375-73d3-40ed-bd85-9996f0995c5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.425295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.425239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.425295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.425274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mht8\" (UniqueName: \"kubernetes.io/projected/03c48375-73d3-40ed-bd85-9996f0995c5f-kube-api-access-8mht8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.425428 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.425310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.525842 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.525808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.525849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c48375-73d3-40ed-bd85-9996f0995c5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.525867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.525888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mht8\" (UniqueName: \"kubernetes.io/projected/03c48375-73d3-40ed-bd85-9996f0995c5f-kube-api-access-8mht8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.525942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.525978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526289 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.526266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526363 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.526308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.526363 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.526342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.528116 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.528097    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c48375-73d3-40ed-bd85-9996f0995c5f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.528376 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.528352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c48375-73d3-40ed-bd85-9996f0995c5f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.533886 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.533866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mht8\" (UniqueName: \"kubernetes.io/projected/03c48375-73d3-40ed-bd85-9996f0995c5f-kube-api-access-8mht8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hwv5c\" (UID: \"03c48375-73d3-40ed-bd85-9996f0995c5f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.644379 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.644354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:43:59.913117 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:43:59.913029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"]
Sep 16 13:43:59.916642 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:43:59.916609    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c48375_73d3_40ed_bd85_9996f0995c5f.slice/crio-351c2afc24bec82d5cd4fe35af726003fb9748cb724970a5284a8199561e283d WatchSource:0}: Error finding container 351c2afc24bec82d5cd4fe35af726003fb9748cb724970a5284a8199561e283d: Status 404 returned error can't find the container with id 351c2afc24bec82d5cd4fe35af726003fb9748cb724970a5284a8199561e283d
Sep 16 13:44:00.276216 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:44:00.276132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c" event={"ID":"03c48375-73d3-40ed-bd85-9996f0995c5f","Type":"ContainerStarted","Data":"8d31e9edf87c162f76022c6380ec668a9d5ae488edefcf36e7cdc057da0f7ec4"}
Sep 16 13:44:00.276216 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:44:00.276181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c" event={"ID":"03c48375-73d3-40ed-bd85-9996f0995c5f","Type":"ContainerStarted","Data":"351c2afc24bec82d5cd4fe35af726003fb9748cb724970a5284a8199561e283d"}
Sep 16 13:44:00.293169 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:44:00.293114    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c" podStartSLOduration=1.111456987 podStartE2EDuration="1.293100671s" totalImagesPullingTime="181.643684ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:44:00.291164595 +0000 UTC m=+667.974822990" watchObservedRunningTime="2026-09-16 13:44:00.293100671 +0000 UTC m=+667.976759065"
Sep 16 13:44:09.644866 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:44:09.644833    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:44:09.645264 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:44:09.645107    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:44:09.657331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:44:09.657310    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:44:10.326914 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:44:10.326881    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hwv5c"
Sep 16 13:45:07.404932 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.404839    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65b68fcbc9-njr2t"]
Sep 16 13:45:07.407932 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.407895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.413877 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.413853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65b68fcbc9-njr2t"]
Sep 16 13:45:07.414691 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.414660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkp9\" (UniqueName: \"kubernetes.io/projected/ed40d17e-2d69-4125-abdf-6f95c4a4e06a-kube-api-access-vpkp9\") pod \"authorino-65b68fcbc9-njr2t\" (UID: \"ed40d17e-2d69-4125-abdf-6f95c4a4e06a\") " pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.414799 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.414713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed40d17e-2d69-4125-abdf-6f95c4a4e06a-tls-cert\") pod \"authorino-65b68fcbc9-njr2t\" (UID: \"ed40d17e-2d69-4125-abdf-6f95c4a4e06a\") " pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.515445 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.515414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed40d17e-2d69-4125-abdf-6f95c4a4e06a-tls-cert\") pod \"authorino-65b68fcbc9-njr2t\" (UID: \"ed40d17e-2d69-4125-abdf-6f95c4a4e06a\") " pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.515588 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.515478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkp9\" (UniqueName: \"kubernetes.io/projected/ed40d17e-2d69-4125-abdf-6f95c4a4e06a-kube-api-access-vpkp9\") pod \"authorino-65b68fcbc9-njr2t\" (UID: \"ed40d17e-2d69-4125-abdf-6f95c4a4e06a\") " pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.517887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.517864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed40d17e-2d69-4125-abdf-6f95c4a4e06a-tls-cert\") pod \"authorino-65b68fcbc9-njr2t\" (UID: \"ed40d17e-2d69-4125-abdf-6f95c4a4e06a\") " pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.523339 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.523317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkp9\" (UniqueName: \"kubernetes.io/projected/ed40d17e-2d69-4125-abdf-6f95c4a4e06a-kube-api-access-vpkp9\") pod \"authorino-65b68fcbc9-njr2t\" (UID: \"ed40d17e-2d69-4125-abdf-6f95c4a4e06a\") " pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.718168 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.718089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65b68fcbc9-njr2t"
Sep 16 13:45:07.983407 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:07.983330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65b68fcbc9-njr2t"]
Sep 16 13:45:07.985891 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:07.985857    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded40d17e_2d69_4125_abdf_6f95c4a4e06a.slice/crio-07b22ec89efb5a575e02ce736a3ba45504676d4428c70fa1290a13ef151e0648 WatchSource:0}: Error finding container 07b22ec89efb5a575e02ce736a3ba45504676d4428c70fa1290a13ef151e0648: Status 404 returned error can't find the container with id 07b22ec89efb5a575e02ce736a3ba45504676d4428c70fa1290a13ef151e0648
Sep 16 13:45:08.553441 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:08.553404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65b68fcbc9-njr2t" event={"ID":"ed40d17e-2d69-4125-abdf-6f95c4a4e06a","Type":"ContainerStarted","Data":"07b22ec89efb5a575e02ce736a3ba45504676d4428c70fa1290a13ef151e0648"}
Sep 16 13:45:09.560140 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.560050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65b68fcbc9-njr2t" event={"ID":"ed40d17e-2d69-4125-abdf-6f95c4a4e06a","Type":"ContainerStarted","Data":"81a91ebf2b0e1780044b7f0054757a1f620bf21763d44802cb9d8957eb8ac151"}
Sep 16 13:45:09.574981 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.574924    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65b68fcbc9-njr2t" podStartSLOduration=1.4171314609999999 podStartE2EDuration="2.574891127s" totalImagesPullingTime="1.157759666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:09.573262396 +0000 UTC m=+737.256920789" watchObservedRunningTime="2026-09-16 13:45:09.574891127 +0000 UTC m=+737.258549563"
Sep 16 13:45:09.601557 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.601488    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-789d6b7b54-xj7mm"]
Sep 16 13:45:09.602410 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.601739    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-789d6b7b54-xj7mm" podUID="1cae7f23-debd-4c2d-b294-1f030764aefd" containerName="authorino" containerID="cri-o://6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23" gracePeriod=30
Sep 16 13:45:09.849864 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.849839    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:45:09.938490 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.938456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1cae7f23-debd-4c2d-b294-1f030764aefd-tls-cert\") pod \"1cae7f23-debd-4c2d-b294-1f030764aefd\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") "
Sep 16 13:45:09.938655 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.938543    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz528\" (UniqueName: \"kubernetes.io/projected/1cae7f23-debd-4c2d-b294-1f030764aefd-kube-api-access-qz528\") pod \"1cae7f23-debd-4c2d-b294-1f030764aefd\" (UID: \"1cae7f23-debd-4c2d-b294-1f030764aefd\") "
Sep 16 13:45:09.940558 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.940523    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cae7f23-debd-4c2d-b294-1f030764aefd-kube-api-access-qz528" (OuterVolumeSpecName: "kube-api-access-qz528") pod "1cae7f23-debd-4c2d-b294-1f030764aefd" (UID: "1cae7f23-debd-4c2d-b294-1f030764aefd"). InnerVolumeSpecName "kube-api-access-qz528". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:09.949027 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:09.948992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cae7f23-debd-4c2d-b294-1f030764aefd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1cae7f23-debd-4c2d-b294-1f030764aefd" (UID: "1cae7f23-debd-4c2d-b294-1f030764aefd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:45:10.039468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.039436    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1cae7f23-debd-4c2d-b294-1f030764aefd-tls-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:45:10.039468 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.039461    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz528\" (UniqueName: \"kubernetes.io/projected/1cae7f23-debd-4c2d-b294-1f030764aefd-kube-api-access-qz528\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:45:10.566092 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.566007    2580 generic.go:356] "Generic (PLEG): container finished" podID="1cae7f23-debd-4c2d-b294-1f030764aefd" containerID="6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23" exitCode=0
Sep 16 13:45:10.566092 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.566058    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-789d6b7b54-xj7mm"
Sep 16 13:45:10.566586 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.566126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789d6b7b54-xj7mm" event={"ID":"1cae7f23-debd-4c2d-b294-1f030764aefd","Type":"ContainerDied","Data":"6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23"}
Sep 16 13:45:10.566586 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.566158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-789d6b7b54-xj7mm" event={"ID":"1cae7f23-debd-4c2d-b294-1f030764aefd","Type":"ContainerDied","Data":"7264c5798006512d7edf745d298a999ee576f51669605d766e4dae8e09d71ae5"}
Sep 16 13:45:10.566586 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.566178    2580 scope.go:117] "RemoveContainer" containerID="6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23"
Sep 16 13:45:10.575574 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.575552    2580 scope.go:117] "RemoveContainer" containerID="6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23"
Sep 16 13:45:10.575845 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:10.575822    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23\": container with ID starting with 6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23 not found: ID does not exist" containerID="6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23"
Sep 16 13:45:10.575924 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.575850    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23"} err="failed to get container status \"6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23\": rpc error: code = NotFound desc = could not find container \"6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23\": container with ID starting with 6c9e58b1704973e63fc6e9ac54938c37339164194c0898360c085201e1e2bf23 not found: ID does not exist"
Sep 16 13:45:10.586105 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.586084    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-789d6b7b54-xj7mm"]
Sep 16 13:45:10.589470 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.589443    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-789d6b7b54-xj7mm"]
Sep 16 13:45:10.802859 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:10.802824    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cae7f23-debd-4c2d-b294-1f030764aefd" path="/var/lib/kubelet/pods/1cae7f23-debd-4c2d-b294-1f030764aefd/volumes"
Sep 16 13:45:31.182029 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.181997    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7704-5cd6c0"]
Sep 16 13:45:31.182384 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.182270    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cae7f23-debd-4c2d-b294-1f030764aefd" containerName="authorino"
Sep 16 13:45:31.182384 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.182280    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cae7f23-debd-4c2d-b294-1f030764aefd" containerName="authorino"
Sep 16 13:45:31.182384 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.182338    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cae7f23-debd-4c2d-b294-1f030764aefd" containerName="authorino"
Sep 16 13:45:31.185059 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.185043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:31.187090 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.187067    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmn8d\""
Sep 16 13:45:31.189591 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.189559    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7704-5cd6c0"]
Sep 16 13:45:31.296322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.296291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lxnh\" (UniqueName: \"kubernetes.io/projected/6cc5b71b-3c7e-4bac-b78a-609baa3074b9-kube-api-access-6lxnh\") pod \"test-curl-7704-5cd6c0\" (UID: \"6cc5b71b-3c7e-4bac-b78a-609baa3074b9\") " pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:31.397522 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.397490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lxnh\" (UniqueName: \"kubernetes.io/projected/6cc5b71b-3c7e-4bac-b78a-609baa3074b9-kube-api-access-6lxnh\") pod \"test-curl-7704-5cd6c0\" (UID: \"6cc5b71b-3c7e-4bac-b78a-609baa3074b9\") " pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:31.404668 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.404646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lxnh\" (UniqueName: \"kubernetes.io/projected/6cc5b71b-3c7e-4bac-b78a-609baa3074b9-kube-api-access-6lxnh\") pod \"test-curl-7704-5cd6c0\" (UID: \"6cc5b71b-3c7e-4bac-b78a-609baa3074b9\") " pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:31.495476 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.495412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:31.761800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:31.761692    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7704-5cd6c0"]
Sep 16 13:45:31.764410 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:31.764384    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cc5b71b_3c7e_4bac_b78a_609baa3074b9.slice/crio-47091cdde0b79969664b2e955610e919d590e8f0e10cb41b50c34c945ac70536 WatchSource:0}: Error finding container 47091cdde0b79969664b2e955610e919d590e8f0e10cb41b50c34c945ac70536: Status 404 returned error can't find the container with id 47091cdde0b79969664b2e955610e919d590e8f0e10cb41b50c34c945ac70536
Sep 16 13:45:32.654919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:32.654869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7704-5cd6c0" event={"ID":"6cc5b71b-3c7e-4bac-b78a-609baa3074b9","Type":"ContainerStarted","Data":"47091cdde0b79969664b2e955610e919d590e8f0e10cb41b50c34c945ac70536"}
Sep 16 13:45:34.665099 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:34.665064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7704-5cd6c0" event={"ID":"6cc5b71b-3c7e-4bac-b78a-609baa3074b9","Type":"ContainerStarted","Data":"e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34"}
Sep 16 13:45:34.678702 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:34.678649    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7704-5cd6c0" podStartSLOduration=1.3673862030000001 podStartE2EDuration="3.678635351s" totalImagesPullingTime="2.311249148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:34.677150585 +0000 UTC m=+762.360808979" watchObservedRunningTime="2026-09-16 13:45:34.678635351 +0000 UTC m=+762.362293746"
Sep 16 13:45:35.024754 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.024674    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"]
Sep 16 13:45:35.027883 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.027866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.030523 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.030505    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-fc8c65-dockercfg-xb5db\""
Sep 16 13:45:35.030996 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.030978    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-fc8c65\""
Sep 16 13:45:35.036827 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.036805    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"]
Sep 16 13:45:35.054483 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.054453    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"]
Sep 16 13:45:35.060503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.060478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.070578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.070552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"]
Sep 16 13:45:35.128602 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.128567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcbfr\" (UniqueName: \"kubernetes.io/projected/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-kube-api-access-mcbfr\") pod \"payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.128602 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.128606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.128786 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.128700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.128786 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.128768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45p4k\" (UniqueName: \"kubernetes.io/projected/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-kube-api-access-45p4k\") pod \"payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.229254 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.229216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcbfr\" (UniqueName: \"kubernetes.io/projected/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-kube-api-access-mcbfr\") pod \"payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.229254 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.229251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.229495 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.229285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.229495 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.229316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45p4k\" (UniqueName: \"kubernetes.io/projected/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-kube-api-access-45p4k\") pod \"payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.230251 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.230227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.230314 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.230227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.241541 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.241517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45p4k\" (UniqueName: \"kubernetes.io/projected/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-kube-api-access-45p4k\") pod \"payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.241649 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.241518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcbfr\" (UniqueName: \"kubernetes.io/projected/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-kube-api-access-mcbfr\") pod \"payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.276403 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.276346    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7704-5cd6c0"]
Sep 16 13:45:35.283780 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.283727    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7704-5cd6c0"]
Sep 16 13:45:35.338549 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.338521    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:35.372440 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.372395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:35.421223 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.421193    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7704-4b26a1"]
Sep 16 13:45:35.421494 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.421480    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" containerName="test-curl-7704-5cd6c0"
Sep 16 13:45:35.421494 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.421493    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" containerName="test-curl-7704-5cd6c0"
Sep 16 13:45:35.421618 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.421553    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" containerName="test-curl-7704-5cd6c0"
Sep 16 13:45:35.424658 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.424623    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:35.428449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.428072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7704-4b26a1"]
Sep 16 13:45:35.531581 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.531506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b9vc\" (UniqueName: \"kubernetes.io/projected/fbe8603f-2b32-472c-9bff-767bfe85656c-kube-api-access-9b9vc\") pod \"test-curl-7704-4b26a1\" (UID: \"fbe8603f-2b32-472c-9bff-767bfe85656c\") " pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:35.615495 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.615462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"]
Sep 16 13:45:35.618346 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:35.618311    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3fc5ef3_057c_41fb_bec4_2a70ca89f058.slice/crio-7d6e65a593eed8898035928e383e4511357d36c92ee83d4981b409793b71bd97 WatchSource:0}: Error finding container 7d6e65a593eed8898035928e383e4511357d36c92ee83d4981b409793b71bd97: Status 404 returned error can't find the container with id 7d6e65a593eed8898035928e383e4511357d36c92ee83d4981b409793b71bd97
Sep 16 13:45:35.632596 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.632574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b9vc\" (UniqueName: \"kubernetes.io/projected/fbe8603f-2b32-472c-9bff-767bfe85656c-kube-api-access-9b9vc\") pod \"test-curl-7704-4b26a1\" (UID: \"fbe8603f-2b32-472c-9bff-767bfe85656c\") " pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:35.639196 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.639176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b9vc\" (UniqueName: \"kubernetes.io/projected/fbe8603f-2b32-472c-9bff-767bfe85656c-kube-api-access-9b9vc\") pod \"test-curl-7704-4b26a1\" (UID: \"fbe8603f-2b32-472c-9bff-767bfe85656c\") " pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:35.653613 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.653525    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"]
Sep 16 13:45:35.655428 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:35.655410    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c70740f_a09c_42c9_a0c3_4ced6bf3d5ab.slice/crio-d400aeeb899af08760d366fa3a117454d348bf48ec54550cd73715c3673a1885 WatchSource:0}: Error finding container d400aeeb899af08760d366fa3a117454d348bf48ec54550cd73715c3673a1885: Status 404 returned error can't find the container with id d400aeeb899af08760d366fa3a117454d348bf48ec54550cd73715c3673a1885
Sep 16 13:45:35.671141 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.671114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r" event={"ID":"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab","Type":"ContainerStarted","Data":"d400aeeb899af08760d366fa3a117454d348bf48ec54550cd73715c3673a1885"}
Sep 16 13:45:35.673299 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.673273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c" event={"ID":"f3fc5ef3-057c-41fb-bec4-2a70ca89f058","Type":"ContainerStarted","Data":"7d6e65a593eed8898035928e383e4511357d36c92ee83d4981b409793b71bd97"}
Sep 16 13:45:35.673463 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:35.673449    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7704-5cd6c0" container="test-curl-7704-5cd6c0"
Sep 16 13:45:35.736658 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.736632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:35.954252 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.954220    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"]
Sep 16 13:45:35.958390 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.958373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:35.960613 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.960588    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-401a67-dockercfg-5ftxz\""
Sep 16 13:45:35.960938 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.960922    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-401a67\""
Sep 16 13:45:35.968086 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:35.968065    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"]
Sep 16 13:45:36.013585 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.013528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7704-4b26a1"]
Sep 16 13:45:36.018978 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:36.018947    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbe8603f_2b32_472c_9bff_767bfe85656c.slice/crio-130d50c1e516aa4f06b6b6c3f7ce2ace7206ec4059ec291d491b4661f200f4f4 WatchSource:0}: Error finding container 130d50c1e516aa4f06b6b6c3f7ce2ace7206ec4059ec291d491b4661f200f4f4: Status 404 returned error can't find the container with id 130d50c1e516aa4f06b6b6c3f7ce2ace7206ec4059ec291d491b4661f200f4f4
Sep 16 13:45:36.035204 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.035184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29xq8\" (UniqueName: \"kubernetes.io/projected/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-kube-api-access-29xq8\") pod \"payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:36.035271 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.035225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:36.136003 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.135959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29xq8\" (UniqueName: \"kubernetes.io/projected/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-kube-api-access-29xq8\") pod \"payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:36.136155 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.136029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:36.136957 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.136930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:36.142974 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.142953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29xq8\" (UniqueName: \"kubernetes.io/projected/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-kube-api-access-29xq8\") pod \"payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:36.268874 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.268842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:36.540184 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.540148    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"]
Sep 16 13:45:36.543030 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:36.543009    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8d7119_4f57_46d3_8685_a7d8b12fc7c5.slice/crio-566191d5b36693aa9d2a8cee0d1be02973224ee9bab21a0b768a32c93108efbb WatchSource:0}: Error finding container 566191d5b36693aa9d2a8cee0d1be02973224ee9bab21a0b768a32c93108efbb: Status 404 returned error can't find the container with id 566191d5b36693aa9d2a8cee0d1be02973224ee9bab21a0b768a32c93108efbb
Sep 16 13:45:36.679307 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.679267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r" event={"ID":"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab","Type":"ContainerStarted","Data":"39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f"}
Sep 16 13:45:36.679669 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.679328    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:36.680807 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.680780    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c" event={"ID":"f3fc5ef3-057c-41fb-bec4-2a70ca89f058","Type":"ContainerStarted","Data":"798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f"}
Sep 16 13:45:36.680942 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.680895    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:36.682333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.682309    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7704-4b26a1" event={"ID":"fbe8603f-2b32-472c-9bff-767bfe85656c","Type":"ContainerStarted","Data":"8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d"}
Sep 16 13:45:36.682427 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.682345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7704-4b26a1" event={"ID":"fbe8603f-2b32-472c-9bff-767bfe85656c","Type":"ContainerStarted","Data":"130d50c1e516aa4f06b6b6c3f7ce2ace7206ec4059ec291d491b4661f200f4f4"}
Sep 16 13:45:36.683806 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.683784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2" event={"ID":"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5","Type":"ContainerStarted","Data":"9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302"}
Sep 16 13:45:36.683876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.683828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2" event={"ID":"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5","Type":"ContainerStarted","Data":"566191d5b36693aa9d2a8cee0d1be02973224ee9bab21a0b768a32c93108efbb"}
Sep 16 13:45:36.683876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.683854    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7704-5cd6c0" podUID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" containerName="test-curl-7704-5cd6c0" containerID="cri-o://e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34" gracePeriod=2
Sep 16 13:45:36.695555 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.695512    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r" podStartSLOduration=1.695496302 podStartE2EDuration="1.695496302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:36.694027555 +0000 UTC m=+764.377685974" watchObservedRunningTime="2026-09-16 13:45:36.695496302 +0000 UTC m=+764.379154696"
Sep 16 13:45:36.695717 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:36.695691    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7704-5cd6c0\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:36.715392 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.715355    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2" podStartSLOduration=1.7153455260000001 podStartE2EDuration="1.715345526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:36.714408599 +0000 UTC m=+764.398066993" watchObservedRunningTime="2026-09-16 13:45:36.715345526 +0000 UTC m=+764.399003913"
Sep 16 13:45:36.727363 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.727330    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7704-4b26a1" podStartSLOduration=1.727319347 podStartE2EDuration="1.727319347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:36.726296195 +0000 UTC m=+764.409954590" watchObservedRunningTime="2026-09-16 13:45:36.727319347 +0000 UTC m=+764.410977741"
Sep 16 13:45:36.739859 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:36.739823    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c" podStartSLOduration=1.739812557 podStartE2EDuration="1.739812557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:36.738628114 +0000 UTC m=+764.422286508" watchObservedRunningTime="2026-09-16 13:45:36.739812557 +0000 UTC m=+764.423470950"
Sep 16 13:45:37.090224 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.090188    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"]
Sep 16 13:45:37.093613 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.093596    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.096008 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.095985    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-7d96c7-dockercfg-p82r7\""
Sep 16 13:45:37.096129 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.096113    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-7d96c7\""
Sep 16 13:45:37.106232 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.106210    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"]
Sep 16 13:45:37.113565 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.113545    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"]
Sep 16 13:45:37.116644 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.116626    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.125771 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.125749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"]
Sep 16 13:45:37.143203 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.143185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn5p8\" (UniqueName: \"kubernetes.io/projected/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-kube-api-access-rn5p8\") pod \"payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.143277 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.143227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.244277 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.244245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb9mm\" (UniqueName: \"kubernetes.io/projected/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-kube-api-access-mb9mm\") pod \"payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.244421 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.244295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.244475 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.244432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn5p8\" (UniqueName: \"kubernetes.io/projected/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-kube-api-access-rn5p8\") pod \"payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.244532 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.244516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.245245 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.245227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.252946 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.252893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn5p8\" (UniqueName: \"kubernetes.io/projected/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-kube-api-access-rn5p8\") pod \"payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.345110 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.345077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.345267 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.345176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb9mm\" (UniqueName: \"kubernetes.io/projected/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-kube-api-access-mb9mm\") pod \"payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.345768 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.345748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.352561 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.352536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb9mm\" (UniqueName: \"kubernetes.io/projected/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-kube-api-access-mb9mm\") pod \"payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.390345 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.390315    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7704-4b26a1"]
Sep 16 13:45:37.396170 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.396103    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7704-4b26a1"]
Sep 16 13:45:37.405722 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.405698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:37.428028 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.428005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:37.685876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.685851    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"]
Sep 16 13:45:37.691058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.688979    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:37.691058 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:37.689504    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7704-4b26a1" container="test-curl-7704-4b26a1"
Sep 16 13:45:37.691507 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:37.691480    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3ee91c_2ac1_4136_af65_d66ceeb8833a.slice/crio-ecceb7bad655a132e2455157aaecc5f26000aa458a7d77e4a2df19ac313190bc WatchSource:0}: Error finding container ecceb7bad655a132e2455157aaecc5f26000aa458a7d77e4a2df19ac313190bc: Status 404 returned error can't find the container with id ecceb7bad655a132e2455157aaecc5f26000aa458a7d77e4a2df19ac313190bc
Sep 16 13:45:37.712745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:37.712634    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"]
Sep 16 13:45:37.715861 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:37.715837    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1377bcf_bfce_435a_8fe9_c2a5a394b23a.slice/crio-f27b794b33b5c0763c7fd503cf415d8f03bf75035ce3e7b1c4ed0db47ebedadf WatchSource:0}: Error finding container f27b794b33b5c0763c7fd503cf415d8f03bf75035ce3e7b1c4ed0db47ebedadf: Status 404 returned error can't find the container with id f27b794b33b5c0763c7fd503cf415d8f03bf75035ce3e7b1c4ed0db47ebedadf
Sep 16 13:45:38.696880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.696849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj" event={"ID":"d1377bcf-bfce-435a-8fe9-c2a5a394b23a","Type":"ContainerStarted","Data":"69052424ef4ab70a923d21bc0d151fab5b29243a733e79bc532561a99295e20d"}
Sep 16 13:45:38.697308 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.697211    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:38.697308 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.697246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj" event={"ID":"d1377bcf-bfce-435a-8fe9-c2a5a394b23a","Type":"ContainerStarted","Data":"f27b794b33b5c0763c7fd503cf415d8f03bf75035ce3e7b1c4ed0db47ebedadf"}
Sep 16 13:45:38.699126 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.699084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l" event={"ID":"6b3ee91c-2ac1-4136-af65-d66ceeb8833a","Type":"ContainerStarted","Data":"f6a168a1c35e2f29097b0e367f468300c56b64324c1d2da7e61b68f546cb6d97"}
Sep 16 13:45:38.699203 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.699131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l" event={"ID":"6b3ee91c-2ac1-4136-af65-d66ceeb8833a","Type":"ContainerStarted","Data":"ecceb7bad655a132e2455157aaecc5f26000aa458a7d77e4a2df19ac313190bc"}
Sep 16 13:45:38.699203 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.699122    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7704-4b26a1" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" containerName="test-curl-7704-4b26a1" containerID="cri-o://8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d" gracePeriod=2
Sep 16 13:45:38.715198 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.714877    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj" podStartSLOduration=1.714857404 podStartE2EDuration="1.714857404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:38.714231769 +0000 UTC m=+766.397890163" watchObservedRunningTime="2026-09-16 13:45:38.714857404 +0000 UTC m=+766.398515928"
Sep 16 13:45:38.731707 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.731661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l" podStartSLOduration=1.731646563 podStartE2EDuration="1.731646563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:38.731395461 +0000 UTC m=+766.415053858" watchObservedRunningTime="2026-09-16 13:45:38.731646563 +0000 UTC m=+766.415304957"
Sep 16 13:45:38.733548 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:38.733520    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7704-4b26a1\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:38.838404 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.838378    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:38.964101 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.964010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lxnh\" (UniqueName: \"kubernetes.io/projected/6cc5b71b-3c7e-4bac-b78a-609baa3074b9-kube-api-access-6lxnh\") pod \"6cc5b71b-3c7e-4bac-b78a-609baa3074b9\" (UID: \"6cc5b71b-3c7e-4bac-b78a-609baa3074b9\") "
Sep 16 13:45:38.966659 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:38.966631    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cc5b71b-3c7e-4bac-b78a-609baa3074b9-kube-api-access-6lxnh" (OuterVolumeSpecName: "kube-api-access-6lxnh") pod "6cc5b71b-3c7e-4bac-b78a-609baa3074b9" (UID: "6cc5b71b-3c7e-4bac-b78a-609baa3074b9"). InnerVolumeSpecName "kube-api-access-6lxnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:39.065345 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:39.065307    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lxnh\" (UniqueName: \"kubernetes.io/projected/6cc5b71b-3c7e-4bac-b78a-609baa3074b9-kube-api-access-6lxnh\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:45:39.705354 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:39.705316    2580 generic.go:356] "Generic (PLEG): container finished" podID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" containerID="e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34" exitCode=137
Sep 16 13:45:39.705354 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:39.705356    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:39.705872 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:39.705420    2580 scope.go:117] "RemoveContainer" containerID="e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34"
Sep 16 13:45:39.705872 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:39.705713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:39.714260 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:39.714244    2580 scope.go:117] "RemoveContainer" containerID="e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34"
Sep 16 13:45:39.714530 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:39.714512    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34\": container with ID starting with e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34 not found: ID does not exist" containerID="e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34"
Sep 16 13:45:39.714577 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:39.714535    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34"} err="failed to get container status \"e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34\": rpc error: code = NotFound desc = could not find container \"e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34\": container with ID starting with e9a888a82133e0771c419099235035856b99328c0c498c4e0c2f602792041c34 not found: ID does not exist"
Sep 16 13:45:39.714997 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:39.714962    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7704-5cd6c0\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" pod="opendatahub/test-curl-7704-5cd6c0"
Sep 16 13:45:40.802757 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:40.802724    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cc5b71b-3c7e-4bac-b78a-609baa3074b9" path="/var/lib/kubelet/pods/6cc5b71b-3c7e-4bac-b78a-609baa3074b9/volumes"
Sep 16 13:45:40.876665 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:40.876642    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:40.982549 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:40.982473    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b9vc\" (UniqueName: \"kubernetes.io/projected/fbe8603f-2b32-472c-9bff-767bfe85656c-kube-api-access-9b9vc\") pod \"fbe8603f-2b32-472c-9bff-767bfe85656c\" (UID: \"fbe8603f-2b32-472c-9bff-767bfe85656c\") "
Sep 16 13:45:40.984470 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:40.984446    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe8603f-2b32-472c-9bff-767bfe85656c-kube-api-access-9b9vc" (OuterVolumeSpecName: "kube-api-access-9b9vc") pod "fbe8603f-2b32-472c-9bff-767bfe85656c" (UID: "fbe8603f-2b32-472c-9bff-767bfe85656c"). InnerVolumeSpecName "kube-api-access-9b9vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:45:41.083868 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:41.083837    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b9vc\" (UniqueName: \"kubernetes.io/projected/fbe8603f-2b32-472c-9bff-767bfe85656c-kube-api-access-9b9vc\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:45:41.718232 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:41.718195    2580 generic.go:356] "Generic (PLEG): container finished" podID="fbe8603f-2b32-472c-9bff-767bfe85656c" containerID="8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d" exitCode=137
Sep 16 13:45:41.718434 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:41.718250    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:41.718434 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:41.718301    2580 scope.go:117] "RemoveContainer" containerID="8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d"
Sep 16 13:45:41.727234 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:41.727139    2580 scope.go:117] "RemoveContainer" containerID="8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d"
Sep 16 13:45:41.727499 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:41.727465    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d\": container with ID starting with 8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d not found: ID does not exist" containerID="8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d"
Sep 16 13:45:41.727624 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:41.727506    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d"} err="failed to get container status \"8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d\": rpc error: code = NotFound desc = could not find container \"8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d\": container with ID starting with 8baa66f6493ea0e53b129fe505dca8ea55a3ecc4b2654a4d178068470881323d not found: ID does not exist"
Sep 16 13:45:41.728928 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:41.728889    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7704-4b26a1\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:42.799960 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:45:42.799893    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7704-4b26a1\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" pod="opendatahub/test-curl-7704-4b26a1"
Sep 16 13:45:42.803341 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:42.803313    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" path="/var/lib/kubelet/pods/fbe8603f-2b32-472c-9bff-767bfe85656c/volumes"
Sep 16 13:45:45.011767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.011719    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"]
Sep 16 13:45:45.012298 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.012105    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" containerName="test-curl-7704-4b26a1"
Sep 16 13:45:45.012298 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.012119    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" containerName="test-curl-7704-4b26a1"
Sep 16 13:45:45.012298 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.012181    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbe8603f-2b32-472c-9bff-767bfe85656c" containerName="test-curl-7704-4b26a1"
Sep 16 13:45:45.016593 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.016572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.018791 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.018759    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-d94fd9\""
Sep 16 13:45:45.018937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.018888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-d94fd9-dockercfg-zvcq7\""
Sep 16 13:45:45.025785 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.025763    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"]
Sep 16 13:45:45.050574 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.050538    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"]
Sep 16 13:45:45.053801 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.053779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.064746 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.064721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"]
Sep 16 13:45:45.125114 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.125075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d78b22-b565-4f04-ae13-37d98f78fbb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.125293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.125131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt7b8\" (UniqueName: \"kubernetes.io/projected/f3d78b22-b565-4f04-ae13-37d98f78fbb5-kube-api-access-pt7b8\") pod \"payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.125293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.125193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/861e51d4-101e-4477-8213-1ec3c41a4b8c-plugins-config-volume\") pod \"payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") " pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.125293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.125277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnkj4\" (UniqueName: \"kubernetes.io/projected/861e51d4-101e-4477-8213-1ec3c41a4b8c-kube-api-access-qnkj4\") pod \"payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") " pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.226435 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.226390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d78b22-b565-4f04-ae13-37d98f78fbb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.226595 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.226457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt7b8\" (UniqueName: \"kubernetes.io/projected/f3d78b22-b565-4f04-ae13-37d98f78fbb5-kube-api-access-pt7b8\") pod \"payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.226595 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.226501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/861e51d4-101e-4477-8213-1ec3c41a4b8c-plugins-config-volume\") pod \"payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") " pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.226595 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.226577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnkj4\" (UniqueName: \"kubernetes.io/projected/861e51d4-101e-4477-8213-1ec3c41a4b8c-kube-api-access-qnkj4\") pod \"payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") " pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.227294 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.227270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/861e51d4-101e-4477-8213-1ec3c41a4b8c-plugins-config-volume\") pod \"payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") " pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.227412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.227343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d78b22-b565-4f04-ae13-37d98f78fbb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.237614 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.237587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt7b8\" (UniqueName: \"kubernetes.io/projected/f3d78b22-b565-4f04-ae13-37d98f78fbb5-kube-api-access-pt7b8\") pod \"payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.237725 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.237638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnkj4\" (UniqueName: \"kubernetes.io/projected/861e51d4-101e-4477-8213-1ec3c41a4b8c-kube-api-access-qnkj4\") pod \"payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") " pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.327748 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.327663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.364940 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.364885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:45.623162 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.623068    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"]
Sep 16 13:45:45.637438 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:45.637405    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d78b22_b565_4f04_ae13_37d98f78fbb5.slice/crio-c34a4f6fc93c3f144247b978587b207d6711479b495c3d155adf6761bbcc0d1a WatchSource:0}: Error finding container c34a4f6fc93c3f144247b978587b207d6711479b495c3d155adf6761bbcc0d1a: Status 404 returned error can't find the container with id c34a4f6fc93c3f144247b978587b207d6711479b495c3d155adf6761bbcc0d1a
Sep 16 13:45:45.682939 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.679593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"]
Sep 16 13:45:45.694176 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:45.694141    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861e51d4_101e_4477_8213_1ec3c41a4b8c.slice/crio-ba8872333e1e2b8d79149eaeefd8f5f9fa2039c2e1224242593317ac995c61fd WatchSource:0}: Error finding container ba8872333e1e2b8d79149eaeefd8f5f9fa2039c2e1224242593317ac995c61fd: Status 404 returned error can't find the container with id ba8872333e1e2b8d79149eaeefd8f5f9fa2039c2e1224242593317ac995c61fd
Sep 16 13:45:45.742339 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.742295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4" event={"ID":"861e51d4-101e-4477-8213-1ec3c41a4b8c","Type":"ContainerStarted","Data":"ba8872333e1e2b8d79149eaeefd8f5f9fa2039c2e1224242593317ac995c61fd"}
Sep 16 13:45:45.744520 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.744488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb" event={"ID":"f3d78b22-b565-4f04-ae13-37d98f78fbb5","Type":"ContainerStarted","Data":"cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef"}
Sep 16 13:45:45.744645 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.744543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb" event={"ID":"f3d78b22-b565-4f04-ae13-37d98f78fbb5","Type":"ContainerStarted","Data":"c34a4f6fc93c3f144247b978587b207d6711479b495c3d155adf6761bbcc0d1a"}
Sep 16 13:45:45.744645 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.744623    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:45.760526 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:45.760464    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb" podStartSLOduration=1.760443162 podStartE2EDuration="1.760443162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:45.759403329 +0000 UTC m=+773.443061723" watchObservedRunningTime="2026-09-16 13:45:45.760443162 +0000 UTC m=+773.444101555"
Sep 16 13:45:46.068633 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.068593    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"]
Sep 16 13:45:46.072244 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.072220    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"]
Sep 16 13:45:46.072384 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.072342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.074591 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.074565    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a2455826-dockercfg-5xd7d\""
Sep 16 13:45:46.074841 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.074826    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a2455826\""
Sep 16 13:45:46.075287 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.075269    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.083331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.083312    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"]
Sep 16 13:45:46.107343 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.107311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"]
Sep 16 13:45:46.237126 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.237094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.237295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.237148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-plugins-config-volume\") pod \"payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") " pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.237295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.237199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhxwt\" (UniqueName: \"kubernetes.io/projected/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-kube-api-access-fhxwt\") pod \"payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.237383 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.237321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkp4s\" (UniqueName: \"kubernetes.io/projected/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-kube-api-access-qkp4s\") pod \"payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") " pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.338056 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.337964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhxwt\" (UniqueName: \"kubernetes.io/projected/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-kube-api-access-fhxwt\") pod \"payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.338056 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.338028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkp4s\" (UniqueName: \"kubernetes.io/projected/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-kube-api-access-qkp4s\") pod \"payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") " pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.338273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.338059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.338273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.338082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-plugins-config-volume\") pod \"payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") " pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.339053 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.339029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.339131 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.339029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-plugins-config-volume\") pod \"payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") " pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.346220 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.346191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhxwt\" (UniqueName: \"kubernetes.io/projected/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-kube-api-access-fhxwt\") pod \"payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.346353 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.346278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkp4s\" (UniqueName: \"kubernetes.io/projected/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-kube-api-access-qkp4s\") pod \"payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") " pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.386402 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.386365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:46.392785 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.392760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:46.673798 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.673748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"]
Sep 16 13:45:46.675743 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:46.675710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e05a7c_45ec_437f_b3dc_bcb9b43b42ee.slice/crio-dc6b799288d123819ebaab08cb7b722e02f7307bffc49fc3fe10d548e5161ec4 WatchSource:0}: Error finding container dc6b799288d123819ebaab08cb7b722e02f7307bffc49fc3fe10d548e5161ec4: Status 404 returned error can't find the container with id dc6b799288d123819ebaab08cb7b722e02f7307bffc49fc3fe10d548e5161ec4
Sep 16 13:45:46.679672 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.679647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"]
Sep 16 13:45:46.682382 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:45:46.682357    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73ed4cc_ea4a_46e6_a757_a6d43b89fa47.slice/crio-d13a4d8add3c11e2df24a0f3ba7114b86fe5b001643ec341794f582c2f5fbbf4 WatchSource:0}: Error finding container d13a4d8add3c11e2df24a0f3ba7114b86fe5b001643ec341794f582c2f5fbbf4: Status 404 returned error can't find the container with id d13a4d8add3c11e2df24a0f3ba7114b86fe5b001643ec341794f582c2f5fbbf4
Sep 16 13:45:46.754624 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.753776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4" event={"ID":"861e51d4-101e-4477-8213-1ec3c41a4b8c","Type":"ContainerStarted","Data":"e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3"}
Sep 16 13:45:46.754624 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.754569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:46.757195 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.757131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk" event={"ID":"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee","Type":"ContainerStarted","Data":"dc6b799288d123819ebaab08cb7b722e02f7307bffc49fc3fe10d548e5161ec4"}
Sep 16 13:45:46.759837 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.759813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2" event={"ID":"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47","Type":"ContainerStarted","Data":"d13a4d8add3c11e2df24a0f3ba7114b86fe5b001643ec341794f582c2f5fbbf4"}
Sep 16 13:45:46.772086 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:46.771929    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4" podStartSLOduration=1.771886588 podStartE2EDuration="1.771886588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:46.771506716 +0000 UTC m=+774.455165124" watchObservedRunningTime="2026-09-16 13:45:46.771886588 +0000 UTC m=+774.455544983"
Sep 16 13:45:47.691024 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:47.690989    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:45:47.691636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:47.691618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:45:47.767081 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:47.767028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk" event={"ID":"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee","Type":"ContainerStarted","Data":"92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826"}
Sep 16 13:45:47.767265 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:47.767125    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:47.768863 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:47.768833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2" event={"ID":"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47","Type":"ContainerStarted","Data":"18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168"}
Sep 16 13:45:47.788697 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:47.788650    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk" podStartSLOduration=1.788634783 podStartE2EDuration="1.788634783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:47.781763905 +0000 UTC m=+775.465422298" watchObservedRunningTime="2026-09-16 13:45:47.788634783 +0000 UTC m=+775.472293177"
Sep 16 13:45:47.801783 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:47.801723    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2" podStartSLOduration=1.801706137 podStartE2EDuration="1.801706137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:45:47.800256327 +0000 UTC m=+775.483914721" watchObservedRunningTime="2026-09-16 13:45:47.801706137 +0000 UTC m=+775.485364530"
Sep 16 13:45:48.701877 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:48.701848    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:45:48.774097 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:48.774059    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:45:49.708123 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:49.708083    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:45:50.716053 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:50.716018    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:45:56.762661 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:56.762632    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:45:58.775533 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:58.775495    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:45:58.775888 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:58.775578    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:45:59.780566 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:45:59.780533    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:46:00.938076 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.938040    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"]
Sep 16 13:46:00.942465 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.942442    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"]
Sep 16 13:46:00.942600 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.942552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:00.944681 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.944656    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-6c7792\""
Sep 16 13:46:00.944798 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.944691    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-6c7792-dockercfg-t7gdk\""
Sep 16 13:46:00.945524 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.945508    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:00.952979 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.952955    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"]
Sep 16 13:46:00.959106 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.958632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"]
Sep 16 13:46:00.966644 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.966620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a62b43f4-4b67-4808-b4b1-12154db1d096-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:00.966745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.966655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prsm7\" (UniqueName: \"kubernetes.io/projected/a62b43f4-4b67-4808-b4b1-12154db1d096-kube-api-access-prsm7\") pod \"payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:00.966745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.966681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwbjl\" (UniqueName: \"kubernetes.io/projected/bfc91291-a734-4134-97cb-c3ae166e4432-kube-api-access-qwbjl\") pod \"payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:00.966861 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:00.966812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc91291-a734-4134-97cb-c3ae166e4432-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:01.068094 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.068059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwbjl\" (UniqueName: \"kubernetes.io/projected/bfc91291-a734-4134-97cb-c3ae166e4432-kube-api-access-qwbjl\") pod \"payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:01.068259 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.068212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc91291-a734-4134-97cb-c3ae166e4432-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:01.068343 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.068258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a62b43f4-4b67-4808-b4b1-12154db1d096-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:01.068343 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.068289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prsm7\" (UniqueName: \"kubernetes.io/projected/a62b43f4-4b67-4808-b4b1-12154db1d096-kube-api-access-prsm7\") pod \"payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:01.068995 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.068975    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc91291-a734-4134-97cb-c3ae166e4432-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:01.069098 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.069070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a62b43f4-4b67-4808-b4b1-12154db1d096-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:01.079292 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.079269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwbjl\" (UniqueName: \"kubernetes.io/projected/bfc91291-a734-4134-97cb-c3ae166e4432-kube-api-access-qwbjl\") pod \"payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:01.079426 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.079352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prsm7\" (UniqueName: \"kubernetes.io/projected/a62b43f4-4b67-4808-b4b1-12154db1d096-kube-api-access-prsm7\") pod \"payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:01.255229 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.255141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:01.263494 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.263470    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:01.535767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.535739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"]
Sep 16 13:46:01.538176 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:46:01.538139    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62b43f4_4b67_4808_b4b1_12154db1d096.slice/crio-464c1cc5cd6c02ba585d69e58b39c4826a1bf8b0c8d54773df9022e9cd214f2a WatchSource:0}: Error finding container 464c1cc5cd6c02ba585d69e58b39c4826a1bf8b0c8d54773df9022e9cd214f2a: Status 404 returned error can't find the container with id 464c1cc5cd6c02ba585d69e58b39c4826a1bf8b0c8d54773df9022e9cd214f2a
Sep 16 13:46:01.550677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.550652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"]
Sep 16 13:46:01.553673 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:46:01.553641    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc91291_a734_4134_97cb_c3ae166e4432.slice/crio-46e354710b1262e92bc40601927d74120565e0be2f78dd675ca948422fbe4830 WatchSource:0}: Error finding container 46e354710b1262e92bc40601927d74120565e0be2f78dd675ca948422fbe4830: Status 404 returned error can't find the container with id 46e354710b1262e92bc40601927d74120565e0be2f78dd675ca948422fbe4830
Sep 16 13:46:01.838153 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.838048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq" event={"ID":"a62b43f4-4b67-4808-b4b1-12154db1d096","Type":"ContainerStarted","Data":"5deb824c119c79f0ae56030f578939305d6c23bd1e40e07adeec94eaa050e6f0"}
Sep 16 13:46:01.838327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.838213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:01.838327 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.838276    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq" event={"ID":"a62b43f4-4b67-4808-b4b1-12154db1d096","Type":"ContainerStarted","Data":"464c1cc5cd6c02ba585d69e58b39c4826a1bf8b0c8d54773df9022e9cd214f2a"}
Sep 16 13:46:01.839789 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.839763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk" event={"ID":"bfc91291-a734-4134-97cb-c3ae166e4432","Type":"ContainerStarted","Data":"a258cc5afe6cda4af485ea951060c346d4a00053d1088f3772993d3c6ac7928d"}
Sep 16 13:46:01.839938 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.839812    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:01.839938 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.839836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk" event={"ID":"bfc91291-a734-4134-97cb-c3ae166e4432","Type":"ContainerStarted","Data":"46e354710b1262e92bc40601927d74120565e0be2f78dd675ca948422fbe4830"}
Sep 16 13:46:01.855018 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.854972    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq" podStartSLOduration=1.8549571440000001 podStartE2EDuration="1.854957144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:46:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:46:01.852872356 +0000 UTC m=+789.536530751" watchObservedRunningTime="2026-09-16 13:46:01.854957144 +0000 UTC m=+789.538615537"
Sep 16 13:46:01.867888 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:01.867834    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk" podStartSLOduration=1.867817525 podStartE2EDuration="1.867817525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:46:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:46:01.867402017 +0000 UTC m=+789.551060414" watchObservedRunningTime="2026-09-16 13:46:01.867817525 +0000 UTC m=+789.551475921"
Sep 16 13:46:12.246304 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.246268    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"]
Sep 16 13:46:12.246873 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.246515    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2" podUID="c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" containerName="payload-processing" containerID="cri-o://18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168" gracePeriod=30
Sep 16 13:46:12.252071 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.252044    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"]
Sep 16 13:46:12.252432 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.252401    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk" podUID="33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" containerName="payload-pre-processing" containerID="cri-o://92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826" gracePeriod=30
Sep 16 13:46:12.585930 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.582852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:46:12.624931 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.621175    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:46:12.669590 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.669557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkp4s\" (UniqueName: \"kubernetes.io/projected/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-kube-api-access-qkp4s\") pod \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") "
Sep 16 13:46:12.669590 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.669594    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-plugins-config-volume\") pod \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\" (UID: \"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47\") "
Sep 16 13:46:12.669590 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.669611    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhxwt\" (UniqueName: \"kubernetes.io/projected/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-kube-api-access-fhxwt\") pod \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") "
Sep 16 13:46:12.669922 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.669728    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-plugins-config-volume\") pod \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\" (UID: \"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee\") "
Sep 16 13:46:12.670202 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.670102    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" (UID: "c73ed4cc-ea4a-46e6-a757-a6d43b89fa47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:46:12.670325 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.670299    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" (UID: "33e05a7c-45ec-437f-b3dc-bcb9b43b42ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:46:12.672486 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.672464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-kube-api-access-qkp4s" (OuterVolumeSpecName: "kube-api-access-qkp4s") pod "c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" (UID: "c73ed4cc-ea4a-46e6-a757-a6d43b89fa47"). InnerVolumeSpecName "kube-api-access-qkp4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:46:12.672486 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.672475    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-kube-api-access-fhxwt" (OuterVolumeSpecName: "kube-api-access-fhxwt") pod "33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" (UID: "33e05a7c-45ec-437f-b3dc-bcb9b43b42ee"). InnerVolumeSpecName "kube-api-access-fhxwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:46:12.770413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.770335    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkp4s\" (UniqueName: \"kubernetes.io/projected/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-kube-api-access-qkp4s\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:12.770413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.770366    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:12.770413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.770378    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhxwt\" (UniqueName: \"kubernetes.io/projected/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-kube-api-access-fhxwt\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:12.770413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.770390    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:12.847080 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.847048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:46:12.847234 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.847136    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:46:12.895817 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.895777    2580 generic.go:356] "Generic (PLEG): container finished" podID="33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" containerID="92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826" exitCode=0
Sep 16 13:46:12.896028 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.895834    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"
Sep 16 13:46:12.896028 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.895881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk" event={"ID":"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee","Type":"ContainerDied","Data":"92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826"}
Sep 16 13:46:12.896028 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.895982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk" event={"ID":"33e05a7c-45ec-437f-b3dc-bcb9b43b42ee","Type":"ContainerDied","Data":"dc6b799288d123819ebaab08cb7b722e02f7307bffc49fc3fe10d548e5161ec4"}
Sep 16 13:46:12.896028 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.896010    2580 scope.go:117] "RemoveContainer" containerID="92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826"
Sep 16 13:46:12.898500 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.898324    2580 generic.go:356] "Generic (PLEG): container finished" podID="c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" containerID="18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168" exitCode=0
Sep 16 13:46:12.898500 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.898385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2" event={"ID":"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47","Type":"ContainerDied","Data":"18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168"}
Sep 16 13:46:12.898500 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.898430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2" event={"ID":"c73ed4cc-ea4a-46e6-a757-a6d43b89fa47","Type":"ContainerDied","Data":"d13a4d8add3c11e2df24a0f3ba7114b86fe5b001643ec341794f582c2f5fbbf4"}
Sep 16 13:46:12.898739 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.898513    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"
Sep 16 13:46:12.909001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.908975    2580 scope.go:117] "RemoveContainer" containerID="92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826"
Sep 16 13:46:12.909326 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:46:12.909299    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826\": container with ID starting with 92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826 not found: ID does not exist" containerID="92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826"
Sep 16 13:46:12.909422 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.909337    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826"} err="failed to get container status \"92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826\": rpc error: code = NotFound desc = could not find container \"92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826\": container with ID starting with 92da6a7b278ab0795135d61657aff08a94dae8d85a3f32ceb79b1f65d6e65826 not found: ID does not exist"
Sep 16 13:46:12.909422 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.909359    2580 scope.go:117] "RemoveContainer" containerID="18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168"
Sep 16 13:46:12.916234 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.916156    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"]
Sep 16 13:46:12.919317 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.919208    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a2455826-6c6c99ff7-brwh2"]
Sep 16 13:46:12.919387 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.919347    2580 scope.go:117] "RemoveContainer" containerID="18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168"
Sep 16 13:46:12.919656 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:46:12.919637    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168\": container with ID starting with 18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168 not found: ID does not exist" containerID="18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168"
Sep 16 13:46:12.919726 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.919666    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168"} err="failed to get container status \"18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168\": rpc error: code = NotFound desc = could not find container \"18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168\": container with ID starting with 18e4d4be504d0d231ffe2bf4cf357bf13275ea55f80b89171e19d315f5698168 not found: ID does not exist"
Sep 16 13:46:12.929941 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.929921    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"]
Sep 16 13:46:12.942801 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:12.942778    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a2455826-65d97d6f54-tn6nk"]
Sep 16 13:46:14.803092 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:14.803061    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" path="/var/lib/kubelet/pods/33e05a7c-45ec-437f-b3dc-bcb9b43b42ee/volumes"
Sep 16 13:46:14.803443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:14.803420    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" path="/var/lib/kubelet/pods/c73ed4cc-ea4a-46e6-a757-a6d43b89fa47/volumes"
Sep 16 13:46:20.926284 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.926249    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"]
Sep 16 13:46:20.926630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.926598    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" containerName="payload-pre-processing"
Sep 16 13:46:20.926630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.926609    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" containerName="payload-pre-processing"
Sep 16 13:46:20.926630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.926620    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" containerName="payload-processing"
Sep 16 13:46:20.926630 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.926626    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" containerName="payload-processing"
Sep 16 13:46:20.926782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.926682    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e05a7c-45ec-437f-b3dc-bcb9b43b42ee" containerName="payload-pre-processing"
Sep 16 13:46:20.926782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.926701    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73ed4cc-ea4a-46e6-a757-a6d43b89fa47" containerName="payload-processing"
Sep 16 13:46:20.929662 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.929640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:20.931638 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.931620    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b2821608\""
Sep 16 13:46:20.937526 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.937496    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"]
Sep 16 13:46:20.973293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.973261    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"]
Sep 16 13:46:20.976370 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.976342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:20.978410 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.978390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b2821608-dockercfg-qvmmx\""
Sep 16 13:46:20.988135 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:20.988112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"]
Sep 16 13:46:21.041530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.041497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") " pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.041710 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.041565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g75ks\" (UniqueName: \"kubernetes.io/projected/779a093e-41e3-4310-ad7e-a130935d5b7e-kube-api-access-g75ks\") pod \"payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.041710 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.041592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/779a093e-41e3-4310-ad7e-a130935d5b7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.041710 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.041650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncthq\" (UniqueName: \"kubernetes.io/projected/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-kube-api-access-ncthq\") pod \"payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") " pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.142783 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.142745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/779a093e-41e3-4310-ad7e-a130935d5b7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.142783 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.142789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncthq\" (UniqueName: \"kubernetes.io/projected/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-kube-api-access-ncthq\") pod \"payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") " pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.143040 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.142951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") " pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.143112 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.143057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g75ks\" (UniqueName: \"kubernetes.io/projected/779a093e-41e3-4310-ad7e-a130935d5b7e-kube-api-access-g75ks\") pod \"payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.143601 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.143573    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/779a093e-41e3-4310-ad7e-a130935d5b7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.143746 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.143728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") " pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.150059 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.150031    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncthq\" (UniqueName: \"kubernetes.io/projected/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-kube-api-access-ncthq\") pod \"payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") " pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.150206 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.150187    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g75ks\" (UniqueName: \"kubernetes.io/projected/779a093e-41e3-4310-ad7e-a130935d5b7e-kube-api-access-g75ks\") pod \"payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.240564 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.240475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.288245 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.288211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.510633 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.510605    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"]
Sep 16 13:46:21.513010 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:46:21.512979    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779a093e_41e3_4310_ad7e_a130935d5b7e.slice/crio-6f45274d50d7f6d3a0a9ca5ae04f3fc9866a334ab1e5bf00522b8a08dc437903 WatchSource:0}: Error finding container 6f45274d50d7f6d3a0a9ca5ae04f3fc9866a334ab1e5bf00522b8a08dc437903: Status 404 returned error can't find the container with id 6f45274d50d7f6d3a0a9ca5ae04f3fc9866a334ab1e5bf00522b8a08dc437903
Sep 16 13:46:21.567018 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.566992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"]
Sep 16 13:46:21.569828 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:46:21.569800    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4bd85f_e6f2_4351_85d7_ab7af48fe36d.slice/crio-09e1ae6eb0f80a1567cdc7e53010d98ec2dd6336706d9e0cc185846c0fbe97bc WatchSource:0}: Error finding container 09e1ae6eb0f80a1567cdc7e53010d98ec2dd6336706d9e0cc185846c0fbe97bc: Status 404 returned error can't find the container with id 09e1ae6eb0f80a1567cdc7e53010d98ec2dd6336706d9e0cc185846c0fbe97bc
Sep 16 13:46:21.948387 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.948344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt" event={"ID":"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d","Type":"ContainerStarted","Data":"5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5"}
Sep 16 13:46:21.948841 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.948406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt" event={"ID":"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d","Type":"ContainerStarted","Data":"09e1ae6eb0f80a1567cdc7e53010d98ec2dd6336706d9e0cc185846c0fbe97bc"}
Sep 16 13:46:21.948841 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.948434    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:21.950118 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.950087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss" event={"ID":"779a093e-41e3-4310-ad7e-a130935d5b7e","Type":"ContainerStarted","Data":"3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d"}
Sep 16 13:46:21.950229 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.950130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss" event={"ID":"779a093e-41e3-4310-ad7e-a130935d5b7e","Type":"ContainerStarted","Data":"6f45274d50d7f6d3a0a9ca5ae04f3fc9866a334ab1e5bf00522b8a08dc437903"}
Sep 16 13:46:21.950229 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.950150    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:21.962555 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.962501    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt" podStartSLOduration=1.9624845990000002 podStartE2EDuration="1.962484599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:46:21.961993842 +0000 UTC m=+809.645652235" watchObservedRunningTime="2026-09-16 13:46:21.962484599 +0000 UTC m=+809.646142994"
Sep 16 13:46:21.983576 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:21.983538    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss" podStartSLOduration=1.9835255250000001 podStartE2EDuration="1.983525525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:46:21.981190399 +0000 UTC m=+809.664849058" watchObservedRunningTime="2026-09-16 13:46:21.983525525 +0000 UTC m=+809.667183918"
Sep 16 13:46:32.958560 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:32.958532    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:32.959014 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:32.958594    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:58.113553 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.113515    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"]
Sep 16 13:46:58.114040 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.113743    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt" podUID="bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" containerName="payload-processing" containerID="cri-o://5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5" gracePeriod=30
Sep 16 13:46:58.120871 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.120837    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"]
Sep 16 13:46:58.121455 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.121126    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss" podUID="779a093e-41e3-4310-ad7e-a130935d5b7e" containerName="payload-pre-processing" containerID="cri-o://3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d" gracePeriod=30
Sep 16 13:46:58.397319 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.397235    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:58.402814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.402731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:58.464324 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.464290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g75ks\" (UniqueName: \"kubernetes.io/projected/779a093e-41e3-4310-ad7e-a130935d5b7e-kube-api-access-g75ks\") pod \"779a093e-41e3-4310-ad7e-a130935d5b7e\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") "
Sep 16 13:46:58.464497 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.464336    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncthq\" (UniqueName: \"kubernetes.io/projected/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-kube-api-access-ncthq\") pod \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") "
Sep 16 13:46:58.464497 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.464356    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-plugins-config-volume\") pod \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\" (UID: \"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d\") "
Sep 16 13:46:58.464497 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.464380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/779a093e-41e3-4310-ad7e-a130935d5b7e-plugins-config-volume\") pod \"779a093e-41e3-4310-ad7e-a130935d5b7e\" (UID: \"779a093e-41e3-4310-ad7e-a130935d5b7e\") "
Sep 16 13:46:58.464807 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.464781    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" (UID: "bd4bd85f-e6f2-4351-85d7-ab7af48fe36d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:46:58.464876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.464785    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/779a093e-41e3-4310-ad7e-a130935d5b7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "779a093e-41e3-4310-ad7e-a130935d5b7e" (UID: "779a093e-41e3-4310-ad7e-a130935d5b7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:46:58.466415 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.466390    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-kube-api-access-ncthq" (OuterVolumeSpecName: "kube-api-access-ncthq") pod "bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" (UID: "bd4bd85f-e6f2-4351-85d7-ab7af48fe36d"). InnerVolumeSpecName "kube-api-access-ncthq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:46:58.466415 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.466404    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779a093e-41e3-4310-ad7e-a130935d5b7e-kube-api-access-g75ks" (OuterVolumeSpecName: "kube-api-access-g75ks") pod "779a093e-41e3-4310-ad7e-a130935d5b7e" (UID: "779a093e-41e3-4310-ad7e-a130935d5b7e"). InnerVolumeSpecName "kube-api-access-g75ks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:46:58.565504 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.565472    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncthq\" (UniqueName: \"kubernetes.io/projected/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-kube-api-access-ncthq\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:58.565504 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.565500    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:58.565504 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.565509    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/779a093e-41e3-4310-ad7e-a130935d5b7e-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:58.565715 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:58.565518    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g75ks\" (UniqueName: \"kubernetes.io/projected/779a093e-41e3-4310-ad7e-a130935d5b7e-kube-api-access-g75ks\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:46:59.141353 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.141315    2580 generic.go:356] "Generic (PLEG): container finished" podID="bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" containerID="5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5" exitCode=0
Sep 16 13:46:59.141734 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.141362    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"
Sep 16 13:46:59.141734 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.141396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt" event={"ID":"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d","Type":"ContainerDied","Data":"5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5"}
Sep 16 13:46:59.141734 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.141452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt" event={"ID":"bd4bd85f-e6f2-4351-85d7-ab7af48fe36d","Type":"ContainerDied","Data":"09e1ae6eb0f80a1567cdc7e53010d98ec2dd6336706d9e0cc185846c0fbe97bc"}
Sep 16 13:46:59.141734 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.141483    2580 scope.go:117] "RemoveContainer" containerID="5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5"
Sep 16 13:46:59.143072 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.142995    2580 generic.go:356] "Generic (PLEG): container finished" podID="779a093e-41e3-4310-ad7e-a130935d5b7e" containerID="3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d" exitCode=0
Sep 16 13:46:59.143139 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.143085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss" event={"ID":"779a093e-41e3-4310-ad7e-a130935d5b7e","Type":"ContainerDied","Data":"3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d"}
Sep 16 13:46:59.143139 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.143127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss" event={"ID":"779a093e-41e3-4310-ad7e-a130935d5b7e","Type":"ContainerDied","Data":"6f45274d50d7f6d3a0a9ca5ae04f3fc9866a334ab1e5bf00522b8a08dc437903"}
Sep 16 13:46:59.143249 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.143190    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"
Sep 16 13:46:59.150420 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.150399    2580 scope.go:117] "RemoveContainer" containerID="5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5"
Sep 16 13:46:59.150729 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:46:59.150707    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5\": container with ID starting with 5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5 not found: ID does not exist" containerID="5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5"
Sep 16 13:46:59.150814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.150736    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5"} err="failed to get container status \"5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5\": rpc error: code = NotFound desc = could not find container \"5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5\": container with ID starting with 5794e499d7e9aadd6f6fc64c56c5afafbf087ac84b7f30f4bbc41c1d62830bd5 not found: ID does not exist"
Sep 16 13:46:59.150814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.150752    2580 scope.go:117] "RemoveContainer" containerID="3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d"
Sep 16 13:46:59.158634 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.158612    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"]
Sep 16 13:46:59.158799 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.158781    2580 scope.go:117] "RemoveContainer" containerID="3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d"
Sep 16 13:46:59.159064 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:46:59.159048    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d\": container with ID starting with 3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d not found: ID does not exist" containerID="3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d"
Sep 16 13:46:59.159129 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.159068    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d"} err="failed to get container status \"3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d\": rpc error: code = NotFound desc = could not find container \"3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d\": container with ID starting with 3d87ee6c69531cbf0c8fa1a2b8d6e6a08c05c4d5cfe84d3c7879897ec6812a4d not found: ID does not exist"
Sep 16 13:46:59.166657 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.166636    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b2821608-85f7cb7574-6m6ss"]
Sep 16 13:46:59.178782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.178756    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"]
Sep 16 13:46:59.186061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:46:59.182051    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b2821608-d87bc8f74-zq2vt"]
Sep 16 13:47:00.802514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:00.802478    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="779a093e-41e3-4310-ad7e-a130935d5b7e" path="/var/lib/kubelet/pods/779a093e-41e3-4310-ad7e-a130935d5b7e/volumes"
Sep 16 13:47:00.802919 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:00.802843    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" path="/var/lib/kubelet/pods/bd4bd85f-e6f2-4351-85d7-ab7af48fe36d/volumes"
Sep 16 13:47:04.222114 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.222082    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"]
Sep 16 13:47:04.222495 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.222456    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779a093e-41e3-4310-ad7e-a130935d5b7e" containerName="payload-pre-processing"
Sep 16 13:47:04.222495 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.222468    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="779a093e-41e3-4310-ad7e-a130935d5b7e" containerName="payload-pre-processing"
Sep 16 13:47:04.222495 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.222476    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" containerName="payload-processing"
Sep 16 13:47:04.222495 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.222481    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" containerName="payload-processing"
Sep 16 13:47:04.222622 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.222532    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd4bd85f-e6f2-4351-85d7-ab7af48fe36d" containerName="payload-processing"
Sep 16 13:47:04.222622 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.222539    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="779a093e-41e3-4310-ad7e-a130935d5b7e" containerName="payload-pre-processing"
Sep 16 13:47:04.226723 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.226703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.228812 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.228790    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-74678972\""
Sep 16 13:47:04.228812 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.228804    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-74678972-dockercfg-rfs6t\""
Sep 16 13:47:04.234744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.234651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"]
Sep 16 13:47:04.259173 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.259136    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"]
Sep 16 13:47:04.262453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.262430    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.269673 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.269631    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"]
Sep 16 13:47:04.315549 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.315506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e148cf-fd38-4370-ab01-d8a570f0d041-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.315549 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.315544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtkgh\" (UniqueName: \"kubernetes.io/projected/41e148cf-fd38-4370-ab01-d8a570f0d041-kube-api-access-dtkgh\") pod \"payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.315761 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.315612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ed55c4-5e64-4761-96d9-06a3990cf5ac-plugins-config-volume\") pod \"payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") " pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.315761 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.315634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgq4b\" (UniqueName: \"kubernetes.io/projected/66ed55c4-5e64-4761-96d9-06a3990cf5ac-kube-api-access-fgq4b\") pod \"payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") " pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.416506 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.416466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e148cf-fd38-4370-ab01-d8a570f0d041-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.416506 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.416505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtkgh\" (UniqueName: \"kubernetes.io/projected/41e148cf-fd38-4370-ab01-d8a570f0d041-kube-api-access-dtkgh\") pod \"payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.416717 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.416564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ed55c4-5e64-4761-96d9-06a3990cf5ac-plugins-config-volume\") pod \"payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") " pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.416717 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.416587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgq4b\" (UniqueName: \"kubernetes.io/projected/66ed55c4-5e64-4761-96d9-06a3990cf5ac-kube-api-access-fgq4b\") pod \"payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") " pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.417331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.417310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e148cf-fd38-4370-ab01-d8a570f0d041-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.417406 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.417388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ed55c4-5e64-4761-96d9-06a3990cf5ac-plugins-config-volume\") pod \"payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") " pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.423644 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.423616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtkgh\" (UniqueName: \"kubernetes.io/projected/41e148cf-fd38-4370-ab01-d8a570f0d041-kube-api-access-dtkgh\") pod \"payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.423769 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.423744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgq4b\" (UniqueName: \"kubernetes.io/projected/66ed55c4-5e64-4761-96d9-06a3990cf5ac-kube-api-access-fgq4b\") pod \"payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") " pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.545147 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.545060    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:04.574267 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.574236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:04.820497 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.820473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"]
Sep 16 13:47:04.822336 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:47:04.822308    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e148cf_fd38_4370_ab01_d8a570f0d041.slice/crio-f577981350a351b308e82fa4b4a5f2fbc732a9ed34e4112ac4919d9c62be675b WatchSource:0}: Error finding container f577981350a351b308e82fa4b4a5f2fbc732a9ed34e4112ac4919d9c62be675b: Status 404 returned error can't find the container with id f577981350a351b308e82fa4b4a5f2fbc732a9ed34e4112ac4919d9c62be675b
Sep 16 13:47:04.855879 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:04.855853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"]
Sep 16 13:47:04.859175 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:47:04.859137    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ed55c4_5e64_4761_96d9_06a3990cf5ac.slice/crio-bd00ada553a49d99a74a550bc3d2db6b923e2b9705a4710789d473c45ce632f7 WatchSource:0}: Error finding container bd00ada553a49d99a74a550bc3d2db6b923e2b9705a4710789d473c45ce632f7: Status 404 returned error can't find the container with id bd00ada553a49d99a74a550bc3d2db6b923e2b9705a4710789d473c45ce632f7
Sep 16 13:47:05.174392 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.174355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb" event={"ID":"66ed55c4-5e64-4761-96d9-06a3990cf5ac","Type":"ContainerStarted","Data":"c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab"}
Sep 16 13:47:05.174571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.174414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb" event={"ID":"66ed55c4-5e64-4761-96d9-06a3990cf5ac","Type":"ContainerStarted","Data":"bd00ada553a49d99a74a550bc3d2db6b923e2b9705a4710789d473c45ce632f7"}
Sep 16 13:47:05.174571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.174518    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:05.176004 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.175977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg" event={"ID":"41e148cf-fd38-4370-ab01-d8a570f0d041","Type":"ContainerStarted","Data":"94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde"}
Sep 16 13:47:05.176109 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.176018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg" event={"ID":"41e148cf-fd38-4370-ab01-d8a570f0d041","Type":"ContainerStarted","Data":"f577981350a351b308e82fa4b4a5f2fbc732a9ed34e4112ac4919d9c62be675b"}
Sep 16 13:47:05.176109 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.176038    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:05.187916 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.187853    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb" podStartSLOduration=1.187837917 podStartE2EDuration="1.187837917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:47:05.187657864 +0000 UTC m=+852.871316259" watchObservedRunningTime="2026-09-16 13:47:05.187837917 +0000 UTC m=+852.871496310"
Sep 16 13:47:05.203988 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:05.203947    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg" podStartSLOduration=1.203934434 podStartE2EDuration="1.203934434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:47:05.202166234 +0000 UTC m=+852.885824631" watchObservedRunningTime="2026-09-16 13:47:05.203934434 +0000 UTC m=+852.887592828"
Sep 16 13:47:16.183040 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:16.182997    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:16.183515 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:16.183081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:29.639736 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.639701    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"]
Sep 16 13:47:29.644135 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.644110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.647020 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.646991    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-d94fd9\"/\"default-dockercfg-q95lw\""
Sep 16 13:47:29.647157 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.646996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-d94fd9\"/\"openshift-service-ca.crt\""
Sep 16 13:47:29.647214 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.647037    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-d94fd9\"/\"autores-model-d94fd9-kserve-self-signed-certs\""
Sep 16 13:47:29.647300 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.647277    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-d94fd9\"/\"kube-root-ca.crt\""
Sep 16 13:47:29.652573 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.652534    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"]
Sep 16 13:47:29.747065 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.747028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v77k2\" (UniqueName: \"kubernetes.io/projected/4fb2f360-247a-4bb1-8206-a81dcb116772-kube-api-access-v77k2\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.747065 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.747063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-home\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.747271 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.747088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-dshm\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.747271 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.747191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-tmp-dir\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.747271 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.747240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-model-cache\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.747271 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.747258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fb2f360-247a-4bb1-8206-a81dcb116772-tls-certs\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.848604 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.848568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fb2f360-247a-4bb1-8206-a81dcb116772-tls-certs\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.848814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.848641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v77k2\" (UniqueName: \"kubernetes.io/projected/4fb2f360-247a-4bb1-8206-a81dcb116772-kube-api-access-v77k2\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.848814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.848672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-home\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.848814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.848707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-dshm\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.848814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.848766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-tmp-dir\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.849102 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.848828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-model-cache\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.849218 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.849189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-home\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.849263 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.849218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-tmp-dir\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.849307 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.849273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-model-cache\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.851076 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.851045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-dshm\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.851273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.851257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fb2f360-247a-4bb1-8206-a81dcb116772-tls-certs\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.855875 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.855848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v77k2\" (UniqueName: \"kubernetes.io/projected/4fb2f360-247a-4bb1-8206-a81dcb116772-kube-api-access-v77k2\") pod \"autores-model-d94fd9-kserve-67f7986dbc-7xdrq\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") " pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:29.957422 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:29.957390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:30.228429 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:30.228404    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"]
Sep 16 13:47:30.230990 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:47:30.230955    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fb2f360_247a_4bb1_8206_a81dcb116772.slice/crio-32019a732c46510451366fa54437ff318a79166e4033dc84edd51fa3e7c27072 WatchSource:0}: Error finding container 32019a732c46510451366fa54437ff318a79166e4033dc84edd51fa3e7c27072: Status 404 returned error can't find the container with id 32019a732c46510451366fa54437ff318a79166e4033dc84edd51fa3e7c27072
Sep 16 13:47:30.300929 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:30.300883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq" event={"ID":"4fb2f360-247a-4bb1-8206-a81dcb116772","Type":"ContainerStarted","Data":"bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4"}
Sep 16 13:47:30.301045 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:30.300946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq" event={"ID":"4fb2f360-247a-4bb1-8206-a81dcb116772","Type":"ContainerStarted","Data":"32019a732c46510451366fa54437ff318a79166e4033dc84edd51fa3e7c27072"}
Sep 16 13:47:30.315887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:30.315832    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq" podStartSLOduration=1.3158116739999999 podStartE2EDuration="1.315811674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:47:30.3153368 +0000 UTC m=+877.998995198" watchObservedRunningTime="2026-09-16 13:47:30.315811674 +0000 UTC m=+877.999470068"
Sep 16 13:47:33.756883 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:33.756846    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"]
Sep 16 13:47:33.757305 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:33.757165    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb" podUID="66ed55c4-5e64-4761-96d9-06a3990cf5ac" containerName="payload-processing" containerID="cri-o://c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab" gracePeriod=30
Sep 16 13:47:33.762347 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:33.762139    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"]
Sep 16 13:47:33.762502 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:33.762355    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg" podUID="41e148cf-fd38-4370-ab01-d8a570f0d041" containerName="payload-pre-processing" containerID="cri-o://94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde" gracePeriod=30
Sep 16 13:47:34.031366 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.031342    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:34.034707 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.034670    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:34.188191 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.188156    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtkgh\" (UniqueName: \"kubernetes.io/projected/41e148cf-fd38-4370-ab01-d8a570f0d041-kube-api-access-dtkgh\") pod \"41e148cf-fd38-4370-ab01-d8a570f0d041\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") "
Sep 16 13:47:34.188398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.188243    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ed55c4-5e64-4761-96d9-06a3990cf5ac-plugins-config-volume\") pod \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") "
Sep 16 13:47:34.188398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.188261    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgq4b\" (UniqueName: \"kubernetes.io/projected/66ed55c4-5e64-4761-96d9-06a3990cf5ac-kube-api-access-fgq4b\") pod \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\" (UID: \"66ed55c4-5e64-4761-96d9-06a3990cf5ac\") "
Sep 16 13:47:34.188398 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.188288    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e148cf-fd38-4370-ab01-d8a570f0d041-plugins-config-volume\") pod \"41e148cf-fd38-4370-ab01-d8a570f0d041\" (UID: \"41e148cf-fd38-4370-ab01-d8a570f0d041\") "
Sep 16 13:47:34.188586 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.188547    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66ed55c4-5e64-4761-96d9-06a3990cf5ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66ed55c4-5e64-4761-96d9-06a3990cf5ac" (UID: "66ed55c4-5e64-4761-96d9-06a3990cf5ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:47:34.188691 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.188670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41e148cf-fd38-4370-ab01-d8a570f0d041-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41e148cf-fd38-4370-ab01-d8a570f0d041" (UID: "41e148cf-fd38-4370-ab01-d8a570f0d041"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:47:34.190367 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.190346    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e148cf-fd38-4370-ab01-d8a570f0d041-kube-api-access-dtkgh" (OuterVolumeSpecName: "kube-api-access-dtkgh") pod "41e148cf-fd38-4370-ab01-d8a570f0d041" (UID: "41e148cf-fd38-4370-ab01-d8a570f0d041"). InnerVolumeSpecName "kube-api-access-dtkgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:47:34.190443 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.190374    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66ed55c4-5e64-4761-96d9-06a3990cf5ac-kube-api-access-fgq4b" (OuterVolumeSpecName: "kube-api-access-fgq4b") pod "66ed55c4-5e64-4761-96d9-06a3990cf5ac" (UID: "66ed55c4-5e64-4761-96d9-06a3990cf5ac"). InnerVolumeSpecName "kube-api-access-fgq4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:47:34.289787 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.289704    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtkgh\" (UniqueName: \"kubernetes.io/projected/41e148cf-fd38-4370-ab01-d8a570f0d041-kube-api-access-dtkgh\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:47:34.289787 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.289733    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ed55c4-5e64-4761-96d9-06a3990cf5ac-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:47:34.289787 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.289743    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgq4b\" (UniqueName: \"kubernetes.io/projected/66ed55c4-5e64-4761-96d9-06a3990cf5ac-kube-api-access-fgq4b\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:47:34.289787 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.289751    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e148cf-fd38-4370-ab01-d8a570f0d041-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:47:34.321104 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.321073    2580 generic.go:356] "Generic (PLEG): container finished" podID="66ed55c4-5e64-4761-96d9-06a3990cf5ac" containerID="c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab" exitCode=0
Sep 16 13:47:34.321281 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.321117    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"
Sep 16 13:47:34.321281 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.321157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb" event={"ID":"66ed55c4-5e64-4761-96d9-06a3990cf5ac","Type":"ContainerDied","Data":"c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab"}
Sep 16 13:47:34.321281 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.321220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb" event={"ID":"66ed55c4-5e64-4761-96d9-06a3990cf5ac","Type":"ContainerDied","Data":"bd00ada553a49d99a74a550bc3d2db6b923e2b9705a4710789d473c45ce632f7"}
Sep 16 13:47:34.321281 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.321255    2580 scope.go:117] "RemoveContainer" containerID="c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab"
Sep 16 13:47:34.322582 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.322550    2580 generic.go:356] "Generic (PLEG): container finished" podID="41e148cf-fd38-4370-ab01-d8a570f0d041" containerID="94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde" exitCode=0
Sep 16 13:47:34.322720 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.322586    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"
Sep 16 13:47:34.322720 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.322597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg" event={"ID":"41e148cf-fd38-4370-ab01-d8a570f0d041","Type":"ContainerDied","Data":"94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde"}
Sep 16 13:47:34.322720 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.322635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg" event={"ID":"41e148cf-fd38-4370-ab01-d8a570f0d041","Type":"ContainerDied","Data":"f577981350a351b308e82fa4b4a5f2fbc732a9ed34e4112ac4919d9c62be675b"}
Sep 16 13:47:34.331377 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.331320    2580 scope.go:117] "RemoveContainer" containerID="c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab"
Sep 16 13:47:34.331659 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:47:34.331640    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab\": container with ID starting with c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab not found: ID does not exist" containerID="c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab"
Sep 16 13:47:34.331721 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.331670    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab"} err="failed to get container status \"c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab\": rpc error: code = NotFound desc = could not find container \"c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab\": container with ID starting with c7b6d2176aea6790ee8440b5d8964a7f402e8bf7c36673b5df17a059c5c50fab not found: ID does not exist"
Sep 16 13:47:34.331721 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.331690    2580 scope.go:117] "RemoveContainer" containerID="94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde"
Sep 16 13:47:34.339479 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.339463    2580 scope.go:117] "RemoveContainer" containerID="94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde"
Sep 16 13:47:34.339732 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:47:34.339716    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde\": container with ID starting with 94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde not found: ID does not exist" containerID="94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde"
Sep 16 13:47:34.339775 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.339739    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde"} err="failed to get container status \"94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde\": rpc error: code = NotFound desc = could not find container \"94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde\": container with ID starting with 94dbee4bf7b7f6bab76b1dcdd108db1bb1a2e9bffe13592c868fdd8c8d760fde not found: ID does not exist"
Sep 16 13:47:34.344884 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.344859    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"]
Sep 16 13:47:34.348843 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.348821    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-74678972-68f87d4666-2ddrg"]
Sep 16 13:47:34.360677 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.360654    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"]
Sep 16 13:47:34.367004 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.366982    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-74678972-68cb76cd55-t9bcb"]
Sep 16 13:47:34.802943 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.802890    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41e148cf-fd38-4370-ab01-d8a570f0d041" path="/var/lib/kubelet/pods/41e148cf-fd38-4370-ab01-d8a570f0d041/volumes"
Sep 16 13:47:34.803307 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:34.803284    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66ed55c4-5e64-4761-96d9-06a3990cf5ac" path="/var/lib/kubelet/pods/66ed55c4-5e64-4761-96d9-06a3990cf5ac/volumes"
Sep 16 13:47:39.957876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:39.957841    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:39.957876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:39.957886    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:39.970741 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:39.970711    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:40.364270 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:40.364194    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:47:42.617646 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.617610    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"]
Sep 16 13:47:42.618870 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.618842    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41e148cf-fd38-4370-ab01-d8a570f0d041" containerName="payload-pre-processing"
Sep 16 13:47:42.619043 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.619031    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41e148cf-fd38-4370-ab01-d8a570f0d041" containerName="payload-pre-processing"
Sep 16 13:47:42.619147 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.619137    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ed55c4-5e64-4761-96d9-06a3990cf5ac" containerName="payload-processing"
Sep 16 13:47:42.619215 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.619207    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ed55c4-5e64-4761-96d9-06a3990cf5ac" containerName="payload-processing"
Sep 16 13:47:42.619437 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.619424    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="41e148cf-fd38-4370-ab01-d8a570f0d041" containerName="payload-pre-processing"
Sep 16 13:47:42.619530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.619521    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ed55c4-5e64-4761-96d9-06a3990cf5ac" containerName="payload-processing"
Sep 16 13:47:42.624502 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.624479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.627255 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.627233    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-8a426531-dockercfg-822bf\""
Sep 16 13:47:42.627373 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.627356    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-8a426531\""
Sep 16 13:47:42.633240 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.633215    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"]
Sep 16 13:47:42.654551 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.654520    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"]
Sep 16 13:47:42.657711 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.657689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:42.660103 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.660083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c15950-1cb4-4048-9eb5-2b5728815639-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.660209 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.660109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgvrt\" (UniqueName: \"kubernetes.io/projected/91c15950-1cb4-4048-9eb5-2b5728815639-kube-api-access-hgvrt\") pod \"payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.669223 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.669199    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"]
Sep 16 13:47:42.761173 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.761139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8zq4\" (UniqueName: \"kubernetes.io/projected/53ce719e-ae95-412a-b683-a84ff5c057da-kube-api-access-b8zq4\") pod \"payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") " pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:42.761333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.761196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c15950-1cb4-4048-9eb5-2b5728815639-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.761333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.761230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgvrt\" (UniqueName: \"kubernetes.io/projected/91c15950-1cb4-4048-9eb5-2b5728815639-kube-api-access-hgvrt\") pod \"payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.761333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.761283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ce719e-ae95-412a-b683-a84ff5c057da-plugins-config-volume\") pod \"payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") " pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:42.762035 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.762013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c15950-1cb4-4048-9eb5-2b5728815639-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.768473 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.768437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgvrt\" (UniqueName: \"kubernetes.io/projected/91c15950-1cb4-4048-9eb5-2b5728815639-kube-api-access-hgvrt\") pod \"payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.862540 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.862503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ce719e-ae95-412a-b683-a84ff5c057da-plugins-config-volume\") pod \"payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") " pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:42.862749 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.862618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8zq4\" (UniqueName: \"kubernetes.io/projected/53ce719e-ae95-412a-b683-a84ff5c057da-kube-api-access-b8zq4\") pod \"payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") " pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:42.863226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.863202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ce719e-ae95-412a-b683-a84ff5c057da-plugins-config-volume\") pod \"payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") " pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:42.872806 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.872743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8zq4\" (UniqueName: \"kubernetes.io/projected/53ce719e-ae95-412a-b683-a84ff5c057da-kube-api-access-b8zq4\") pod \"payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") " pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:42.961620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.961589    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:42.970860 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:42.970830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:43.240931 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.240748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"]
Sep 16 13:47:43.243273 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:47:43.243244    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c15950_1cb4_4048_9eb5_2b5728815639.slice/crio-e69089e54c086f792a9aadec91be9cd8ac5388d2b33234ae88ec86ff0af39a1c WatchSource:0}: Error finding container e69089e54c086f792a9aadec91be9cd8ac5388d2b33234ae88ec86ff0af39a1c: Status 404 returned error can't find the container with id e69089e54c086f792a9aadec91be9cd8ac5388d2b33234ae88ec86ff0af39a1c
Sep 16 13:47:43.258020 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.257995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"]
Sep 16 13:47:43.261350 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:47:43.261328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ce719e_ae95_412a_b683_a84ff5c057da.slice/crio-68600a593bbed7ee84958b9fd9d096ec191c4e11d68b889a6c69810b4d84fc7c WatchSource:0}: Error finding container 68600a593bbed7ee84958b9fd9d096ec191c4e11d68b889a6c69810b4d84fc7c: Status 404 returned error can't find the container with id 68600a593bbed7ee84958b9fd9d096ec191c4e11d68b889a6c69810b4d84fc7c
Sep 16 13:47:43.368181 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.368144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m" event={"ID":"91c15950-1cb4-4048-9eb5-2b5728815639","Type":"ContainerStarted","Data":"f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451"}
Sep 16 13:47:43.368365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.368306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:47:43.368365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.368353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m" event={"ID":"91c15950-1cb4-4048-9eb5-2b5728815639","Type":"ContainerStarted","Data":"e69089e54c086f792a9aadec91be9cd8ac5388d2b33234ae88ec86ff0af39a1c"}
Sep 16 13:47:43.369774 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.369750    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb" event={"ID":"53ce719e-ae95-412a-b683-a84ff5c057da","Type":"ContainerStarted","Data":"9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10"}
Sep 16 13:47:43.369928 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.369794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb" event={"ID":"53ce719e-ae95-412a-b683-a84ff5c057da","Type":"ContainerStarted","Data":"68600a593bbed7ee84958b9fd9d096ec191c4e11d68b889a6c69810b4d84fc7c"}
Sep 16 13:47:43.369928 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.369831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:43.383655 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.383604    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m" podStartSLOduration=1.383586363 podStartE2EDuration="1.383586363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:47:43.38176091 +0000 UTC m=+891.065419304" watchObservedRunningTime="2026-09-16 13:47:43.383586363 +0000 UTC m=+891.067244757"
Sep 16 13:47:43.402482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:43.402441    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb" podStartSLOduration=1.402427957 podStartE2EDuration="1.402427957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:47:43.400473767 +0000 UTC m=+891.084132162" watchObservedRunningTime="2026-09-16 13:47:43.402427957 +0000 UTC m=+891.086086352"
Sep 16 13:47:52.718829 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:52.718750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:47:52.726643 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:52.726621    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:47:54.376097 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:54.376063    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:47:54.376601 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:47:54.376153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:48:05.387808 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.387774    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"]
Sep 16 13:48:05.392659 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.392633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"
Sep 16 13:48:05.395114 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.395087    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:48:05.395618 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.395605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:48:05.399631 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.399600    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"]
Sep 16 13:48:05.460547 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.460508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572f7d95-e185-4b93-84cb-be257a0bd7e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk\" (UID: \"572f7d95-e185-4b93-84cb-be257a0bd7e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"
Sep 16 13:48:05.561803 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.561771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572f7d95-e185-4b93-84cb-be257a0bd7e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk\" (UID: \"572f7d95-e185-4b93-84cb-be257a0bd7e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"
Sep 16 13:48:05.562406 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.562387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572f7d95-e185-4b93-84cb-be257a0bd7e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk\" (UID: \"572f7d95-e185-4b93-84cb-be257a0bd7e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"
Sep 16 13:48:05.704869 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.704801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"
Sep 16 13:48:05.975910 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:05.975772    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"]
Sep 16 13:48:05.978604 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:48:05.978577    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572f7d95_e185_4b93_84cb_be257a0bd7e5.slice/crio-04be7431c8e3b03ac56d569aafb3dec4b08404c5a7220a9096bbd6f31edd9b63 WatchSource:0}: Error finding container 04be7431c8e3b03ac56d569aafb3dec4b08404c5a7220a9096bbd6f31edd9b63: Status 404 returned error can't find the container with id 04be7431c8e3b03ac56d569aafb3dec4b08404c5a7220a9096bbd6f31edd9b63
Sep 16 13:48:06.491001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:06.490965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk" event={"ID":"572f7d95-e185-4b93-84cb-be257a0bd7e5","Type":"ContainerStarted","Data":"04be7431c8e3b03ac56d569aafb3dec4b08404c5a7220a9096bbd6f31edd9b63"}
Sep 16 13:48:09.508838 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:09.508806    2580 generic.go:356] "Generic (PLEG): container finished" podID="572f7d95-e185-4b93-84cb-be257a0bd7e5" containerID="ad2f4fb1d6b65c5a6c50bbc282a3ae0e4f7d664ace1a12549d17d460f5d3c38d" exitCode=0
Sep 16 13:48:09.509251 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:09.508893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk" event={"ID":"572f7d95-e185-4b93-84cb-be257a0bd7e5","Type":"ContainerDied","Data":"ad2f4fb1d6b65c5a6c50bbc282a3ae0e4f7d664ace1a12549d17d460f5d3c38d"}
Sep 16 13:48:10.648522 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:10.648499    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"
Sep 16 13:48:10.710177 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:10.710148    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572f7d95-e185-4b93-84cb-be257a0bd7e5-maas-api-service-ca\") pod \"572f7d95-e185-4b93-84cb-be257a0bd7e5\" (UID: \"572f7d95-e185-4b93-84cb-be257a0bd7e5\") "
Sep 16 13:48:10.710472 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:10.710449    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572f7d95-e185-4b93-84cb-be257a0bd7e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "572f7d95-e185-4b93-84cb-be257a0bd7e5" (UID: "572f7d95-e185-4b93-84cb-be257a0bd7e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:48:10.811194 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:10.811167    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572f7d95-e185-4b93-84cb-be257a0bd7e5-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:10.893972 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:10.893938    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"]
Sep 16 13:48:11.520417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:11.520386    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"
Sep 16 13:48:11.520417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:11.520403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk" event={"ID":"572f7d95-e185-4b93-84cb-be257a0bd7e5","Type":"ContainerDied","Data":"04be7431c8e3b03ac56d569aafb3dec4b08404c5a7220a9096bbd6f31edd9b63"}
Sep 16 13:48:11.520647 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:11.520446    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04be7431c8e3b03ac56d569aafb3dec4b08404c5a7220a9096bbd6f31edd9b63"
Sep 16 13:48:16.161021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.160989    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"]
Sep 16 13:48:16.161454 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.161229    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4" podUID="861e51d4-101e-4477-8213-1ec3c41a4b8c" containerName="payload-processing" containerID="cri-o://e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3" gracePeriod=30
Sep 16 13:48:16.166984 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.166948    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"]
Sep 16 13:48:16.167693 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.167597    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb" podUID="f3d78b22-b565-4f04-ae13-37d98f78fbb5" containerName="payload-pre-processing" containerID="cri-o://cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef" gracePeriod=30
Sep 16 13:48:16.319266 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.319230    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"]
Sep 16 13:48:16.323195 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.323160    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-d94fd9-vpnvk"]
Sep 16 13:48:16.448321 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.448287    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:48:16.451213 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.451189    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:48:16.543430 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.543394    2580 generic.go:356] "Generic (PLEG): container finished" podID="861e51d4-101e-4477-8213-1ec3c41a4b8c" containerID="e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3" exitCode=0
Sep 16 13:48:16.543578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.543453    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"
Sep 16 13:48:16.543578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.543483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4" event={"ID":"861e51d4-101e-4477-8213-1ec3c41a4b8c","Type":"ContainerDied","Data":"e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3"}
Sep 16 13:48:16.543578 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.543568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4" event={"ID":"861e51d4-101e-4477-8213-1ec3c41a4b8c","Type":"ContainerDied","Data":"ba8872333e1e2b8d79149eaeefd8f5f9fa2039c2e1224242593317ac995c61fd"}
Sep 16 13:48:16.543710 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.543596    2580 scope.go:117] "RemoveContainer" containerID="e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3"
Sep 16 13:48:16.544814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.544787    2580 generic.go:356] "Generic (PLEG): container finished" podID="f3d78b22-b565-4f04-ae13-37d98f78fbb5" containerID="cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef" exitCode=0
Sep 16 13:48:16.544935 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.544838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb" event={"ID":"f3d78b22-b565-4f04-ae13-37d98f78fbb5","Type":"ContainerDied","Data":"cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef"}
Sep 16 13:48:16.544935 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.544888    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb" event={"ID":"f3d78b22-b565-4f04-ae13-37d98f78fbb5","Type":"ContainerDied","Data":"c34a4f6fc93c3f144247b978587b207d6711479b495c3d155adf6761bbcc0d1a"}
Sep 16 13:48:16.544935 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.544841    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"
Sep 16 13:48:16.552006 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.551988    2580 scope.go:117] "RemoveContainer" containerID="e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3"
Sep 16 13:48:16.552279 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:48:16.552258    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3\": container with ID starting with e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3 not found: ID does not exist" containerID="e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3"
Sep 16 13:48:16.552328 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.552288    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3"} err="failed to get container status \"e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3\": rpc error: code = NotFound desc = could not find container \"e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3\": container with ID starting with e225559b67d084855d715db212d75b5de1fe46b5d3b43127878ba74eb5f80eb3 not found: ID does not exist"
Sep 16 13:48:16.552328 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.552305    2580 scope.go:117] "RemoveContainer" containerID="cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef"
Sep 16 13:48:16.557393 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.557375    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt7b8\" (UniqueName: \"kubernetes.io/projected/f3d78b22-b565-4f04-ae13-37d98f78fbb5-kube-api-access-pt7b8\") pod \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") "
Sep 16 13:48:16.557449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.557407    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d78b22-b565-4f04-ae13-37d98f78fbb5-plugins-config-volume\") pod \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\" (UID: \"f3d78b22-b565-4f04-ae13-37d98f78fbb5\") "
Sep 16 13:48:16.557449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.557442    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/861e51d4-101e-4477-8213-1ec3c41a4b8c-plugins-config-volume\") pod \"861e51d4-101e-4477-8213-1ec3c41a4b8c\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") "
Sep 16 13:48:16.557519 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.557481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnkj4\" (UniqueName: \"kubernetes.io/projected/861e51d4-101e-4477-8213-1ec3c41a4b8c-kube-api-access-qnkj4\") pod \"861e51d4-101e-4477-8213-1ec3c41a4b8c\" (UID: \"861e51d4-101e-4477-8213-1ec3c41a4b8c\") "
Sep 16 13:48:16.557805 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.557782    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d78b22-b565-4f04-ae13-37d98f78fbb5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3d78b22-b565-4f04-ae13-37d98f78fbb5" (UID: "f3d78b22-b565-4f04-ae13-37d98f78fbb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:48:16.557874 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.557802    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861e51d4-101e-4477-8213-1ec3c41a4b8c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "861e51d4-101e-4477-8213-1ec3c41a4b8c" (UID: "861e51d4-101e-4477-8213-1ec3c41a4b8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:48:16.559552 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.559530    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861e51d4-101e-4477-8213-1ec3c41a4b8c-kube-api-access-qnkj4" (OuterVolumeSpecName: "kube-api-access-qnkj4") pod "861e51d4-101e-4477-8213-1ec3c41a4b8c" (UID: "861e51d4-101e-4477-8213-1ec3c41a4b8c"). InnerVolumeSpecName "kube-api-access-qnkj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:48:16.559798 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.559779    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d78b22-b565-4f04-ae13-37d98f78fbb5-kube-api-access-pt7b8" (OuterVolumeSpecName: "kube-api-access-pt7b8") pod "f3d78b22-b565-4f04-ae13-37d98f78fbb5" (UID: "f3d78b22-b565-4f04-ae13-37d98f78fbb5"). InnerVolumeSpecName "kube-api-access-pt7b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:48:16.560627 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.560608    2580 scope.go:117] "RemoveContainer" containerID="cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef"
Sep 16 13:48:16.560881 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:48:16.560863    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef\": container with ID starting with cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef not found: ID does not exist" containerID="cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef"
Sep 16 13:48:16.560966 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.560895    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef"} err="failed to get container status \"cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef\": rpc error: code = NotFound desc = could not find container \"cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef\": container with ID starting with cf679a897b330d2ffb86f593ed7483602b20d904e3e64ac60d51ad68f74020ef not found: ID does not exist"
Sep 16 13:48:16.658770 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.658744    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt7b8\" (UniqueName: \"kubernetes.io/projected/f3d78b22-b565-4f04-ae13-37d98f78fbb5-kube-api-access-pt7b8\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:16.658770 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.658767    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d78b22-b565-4f04-ae13-37d98f78fbb5-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:16.658926 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.658777    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/861e51d4-101e-4477-8213-1ec3c41a4b8c-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:16.658926 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.658785    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnkj4\" (UniqueName: \"kubernetes.io/projected/861e51d4-101e-4477-8213-1ec3c41a4b8c-kube-api-access-qnkj4\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:16.802582 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.802507    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="572f7d95-e185-4b93-84cb-be257a0bd7e5" path="/var/lib/kubelet/pods/572f7d95-e185-4b93-84cb-be257a0bd7e5/volumes"
Sep 16 13:48:16.860145 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.860119    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"]
Sep 16 13:48:16.863795 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.863774    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-d94fd9-db66c95bf-5bxz4"]
Sep 16 13:48:16.873026 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.873007    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"]
Sep 16 13:48:16.877866 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:16.877846    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-d94fd9-dbc44ccd6-n9bjb"]
Sep 16 13:48:18.802786 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:18.802750    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861e51d4-101e-4477-8213-1ec3c41a4b8c" path="/var/lib/kubelet/pods/861e51d4-101e-4477-8213-1ec3c41a4b8c/volumes"
Sep 16 13:48:18.803173 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:18.803127    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d78b22-b565-4f04-ae13-37d98f78fbb5" path="/var/lib/kubelet/pods/f3d78b22-b565-4f04-ae13-37d98f78fbb5/volumes"
Sep 16 13:48:20.938473 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:20.938442    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"]
Sep 16 13:48:20.938914 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:20.938666    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb" podUID="53ce719e-ae95-412a-b683-a84ff5c057da" containerName="payload-processing" containerID="cri-o://9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10" gracePeriod=30
Sep 16 13:48:20.950566 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:20.950535    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"]
Sep 16 13:48:20.950851 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:20.950790    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m" podUID="91c15950-1cb4-4048-9eb5-2b5728815639" containerName="payload-pre-processing" containerID="cri-o://f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451" gracePeriod=30
Sep 16 13:48:21.224601 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.224541    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:48:21.228056 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.228032    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:48:21.403269 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.403223    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c15950-1cb4-4048-9eb5-2b5728815639-plugins-config-volume\") pod \"91c15950-1cb4-4048-9eb5-2b5728815639\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") "
Sep 16 13:48:21.403431 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.403321    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8zq4\" (UniqueName: \"kubernetes.io/projected/53ce719e-ae95-412a-b683-a84ff5c057da-kube-api-access-b8zq4\") pod \"53ce719e-ae95-412a-b683-a84ff5c057da\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") "
Sep 16 13:48:21.403431 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.403380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgvrt\" (UniqueName: \"kubernetes.io/projected/91c15950-1cb4-4048-9eb5-2b5728815639-kube-api-access-hgvrt\") pod \"91c15950-1cb4-4048-9eb5-2b5728815639\" (UID: \"91c15950-1cb4-4048-9eb5-2b5728815639\") "
Sep 16 13:48:21.403431 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.403419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ce719e-ae95-412a-b683-a84ff5c057da-plugins-config-volume\") pod \"53ce719e-ae95-412a-b683-a84ff5c057da\" (UID: \"53ce719e-ae95-412a-b683-a84ff5c057da\") "
Sep 16 13:48:21.403695 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.403666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c15950-1cb4-4048-9eb5-2b5728815639-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91c15950-1cb4-4048-9eb5-2b5728815639" (UID: "91c15950-1cb4-4048-9eb5-2b5728815639"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:48:21.403913 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.403877    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53ce719e-ae95-412a-b683-a84ff5c057da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53ce719e-ae95-412a-b683-a84ff5c057da" (UID: "53ce719e-ae95-412a-b683-a84ff5c057da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:48:21.405506 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.405486    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91c15950-1cb4-4048-9eb5-2b5728815639-kube-api-access-hgvrt" (OuterVolumeSpecName: "kube-api-access-hgvrt") pod "91c15950-1cb4-4048-9eb5-2b5728815639" (UID: "91c15950-1cb4-4048-9eb5-2b5728815639"). InnerVolumeSpecName "kube-api-access-hgvrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:48:21.405823 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.405802    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53ce719e-ae95-412a-b683-a84ff5c057da-kube-api-access-b8zq4" (OuterVolumeSpecName: "kube-api-access-b8zq4") pod "53ce719e-ae95-412a-b683-a84ff5c057da" (UID: "53ce719e-ae95-412a-b683-a84ff5c057da"). InnerVolumeSpecName "kube-api-access-b8zq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:48:21.504531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.504460    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8zq4\" (UniqueName: \"kubernetes.io/projected/53ce719e-ae95-412a-b683-a84ff5c057da-kube-api-access-b8zq4\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:21.504531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.504482    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgvrt\" (UniqueName: \"kubernetes.io/projected/91c15950-1cb4-4048-9eb5-2b5728815639-kube-api-access-hgvrt\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:21.504531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.504491    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53ce719e-ae95-412a-b683-a84ff5c057da-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:21.504531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.504500    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c15950-1cb4-4048-9eb5-2b5728815639-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:21.570809 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.570780    2580 generic.go:356] "Generic (PLEG): container finished" podID="91c15950-1cb4-4048-9eb5-2b5728815639" containerID="f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451" exitCode=0
Sep 16 13:48:21.570939 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.570849    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"
Sep 16 13:48:21.570939 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.570859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m" event={"ID":"91c15950-1cb4-4048-9eb5-2b5728815639","Type":"ContainerDied","Data":"f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451"}
Sep 16 13:48:21.571045 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.570965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m" event={"ID":"91c15950-1cb4-4048-9eb5-2b5728815639","Type":"ContainerDied","Data":"e69089e54c086f792a9aadec91be9cd8ac5388d2b33234ae88ec86ff0af39a1c"}
Sep 16 13:48:21.571045 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.570991    2580 scope.go:117] "RemoveContainer" containerID="f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451"
Sep 16 13:48:21.572356 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.572334    2580 generic.go:356] "Generic (PLEG): container finished" podID="53ce719e-ae95-412a-b683-a84ff5c057da" containerID="9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10" exitCode=0
Sep 16 13:48:21.572544 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.572378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb" event={"ID":"53ce719e-ae95-412a-b683-a84ff5c057da","Type":"ContainerDied","Data":"9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10"}
Sep 16 13:48:21.572544 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.572385    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"
Sep 16 13:48:21.572544 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.572413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb" event={"ID":"53ce719e-ae95-412a-b683-a84ff5c057da","Type":"ContainerDied","Data":"68600a593bbed7ee84958b9fd9d096ec191c4e11d68b889a6c69810b4d84fc7c"}
Sep 16 13:48:21.580959 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.580938    2580 scope.go:117] "RemoveContainer" containerID="f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451"
Sep 16 13:48:21.581293 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:48:21.581262    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451\": container with ID starting with f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451 not found: ID does not exist" containerID="f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451"
Sep 16 13:48:21.581347 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.581303    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451"} err="failed to get container status \"f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451\": rpc error: code = NotFound desc = could not find container \"f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451\": container with ID starting with f9884987385e943f24053fc4fc34956ea379f24b52511c78d85eda18ef054451 not found: ID does not exist"
Sep 16 13:48:21.581347 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.581324    2580 scope.go:117] "RemoveContainer" containerID="9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10"
Sep 16 13:48:21.589084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.589068    2580 scope.go:117] "RemoveContainer" containerID="9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10"
Sep 16 13:48:21.589361 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:48:21.589334    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10\": container with ID starting with 9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10 not found: ID does not exist" containerID="9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10"
Sep 16 13:48:21.589424 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.589369    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10"} err="failed to get container status \"9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10\": rpc error: code = NotFound desc = could not find container \"9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10\": container with ID starting with 9fba312aecb22951ae0860949c2af7409cceae29dde0d7c017b2fcfa37ef5f10 not found: ID does not exist"
Sep 16 13:48:21.594775 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.594751    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"]
Sep 16 13:48:21.598800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.598779    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8a426531-c8668c79d-mcb5m"]
Sep 16 13:48:21.607472 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.607453    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"]
Sep 16 13:48:21.613525 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:21.613505    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8a426531-5cd7b97465-7mrvb"]
Sep 16 13:48:22.803067 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:22.803026    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53ce719e-ae95-412a-b683-a84ff5c057da" path="/var/lib/kubelet/pods/53ce719e-ae95-412a-b683-a84ff5c057da/volumes"
Sep 16 13:48:22.803453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:22.803434    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91c15950-1cb4-4048-9eb5-2b5728815639" path="/var/lib/kubelet/pods/91c15950-1cb4-4048-9eb5-2b5728815639/volumes"
Sep 16 13:48:25.912943 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:25.912865    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq" podUID="4fb2f360-247a-4bb1-8206-a81dcb116772" containerName="main" containerID="cri-o://bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4" gracePeriod=45
Sep 16 13:48:26.152797 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.152768    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:48:26.343343 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343277    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-dshm\") pod \"4fb2f360-247a-4bb1-8206-a81dcb116772\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") "
Sep 16 13:48:26.343343 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343327    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-model-cache\") pod \"4fb2f360-247a-4bb1-8206-a81dcb116772\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") "
Sep 16 13:48:26.343514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343348    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fb2f360-247a-4bb1-8206-a81dcb116772-tls-certs\") pod \"4fb2f360-247a-4bb1-8206-a81dcb116772\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") "
Sep 16 13:48:26.343514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343392    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v77k2\" (UniqueName: \"kubernetes.io/projected/4fb2f360-247a-4bb1-8206-a81dcb116772-kube-api-access-v77k2\") pod \"4fb2f360-247a-4bb1-8206-a81dcb116772\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") "
Sep 16 13:48:26.343514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343418    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-tmp-dir\") pod \"4fb2f360-247a-4bb1-8206-a81dcb116772\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") "
Sep 16 13:48:26.343514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343452    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-home\") pod \"4fb2f360-247a-4bb1-8206-a81dcb116772\" (UID: \"4fb2f360-247a-4bb1-8206-a81dcb116772\") "
Sep 16 13:48:26.343707 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343581    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-model-cache" (OuterVolumeSpecName: "model-cache") pod "4fb2f360-247a-4bb1-8206-a81dcb116772" (UID: "4fb2f360-247a-4bb1-8206-a81dcb116772"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:48:26.343767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343705    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4fb2f360-247a-4bb1-8206-a81dcb116772" (UID: "4fb2f360-247a-4bb1-8206-a81dcb116772"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:48:26.343767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343715    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:26.343860 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.343784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-home" (OuterVolumeSpecName: "home") pod "4fb2f360-247a-4bb1-8206-a81dcb116772" (UID: "4fb2f360-247a-4bb1-8206-a81dcb116772"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:48:26.345529 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.345500    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fb2f360-247a-4bb1-8206-a81dcb116772-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4fb2f360-247a-4bb1-8206-a81dcb116772" (UID: "4fb2f360-247a-4bb1-8206-a81dcb116772"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:48:26.345645 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.345536    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-dshm" (OuterVolumeSpecName: "dshm") pod "4fb2f360-247a-4bb1-8206-a81dcb116772" (UID: "4fb2f360-247a-4bb1-8206-a81dcb116772"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:48:26.345645 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.345555    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb2f360-247a-4bb1-8206-a81dcb116772-kube-api-access-v77k2" (OuterVolumeSpecName: "kube-api-access-v77k2") pod "4fb2f360-247a-4bb1-8206-a81dcb116772" (UID: "4fb2f360-247a-4bb1-8206-a81dcb116772"). InnerVolumeSpecName "kube-api-access-v77k2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:48:26.444540 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.444512    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fb2f360-247a-4bb1-8206-a81dcb116772-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:26.444540 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.444535    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v77k2\" (UniqueName: \"kubernetes.io/projected/4fb2f360-247a-4bb1-8206-a81dcb116772-kube-api-access-v77k2\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:26.444695 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.444564    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:26.444695 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.444575    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:26.444695 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.444583    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fb2f360-247a-4bb1-8206-a81dcb116772-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:48:26.598583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.598491    2580 generic.go:356] "Generic (PLEG): container finished" podID="4fb2f360-247a-4bb1-8206-a81dcb116772" containerID="bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4" exitCode=0
Sep 16 13:48:26.598583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.598535    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"
Sep 16 13:48:26.598797 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.598577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq" event={"ID":"4fb2f360-247a-4bb1-8206-a81dcb116772","Type":"ContainerDied","Data":"bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4"}
Sep 16 13:48:26.598797 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.598621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq" event={"ID":"4fb2f360-247a-4bb1-8206-a81dcb116772","Type":"ContainerDied","Data":"32019a732c46510451366fa54437ff318a79166e4033dc84edd51fa3e7c27072"}
Sep 16 13:48:26.598797 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.598638    2580 scope.go:117] "RemoveContainer" containerID="bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4"
Sep 16 13:48:26.607034 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.607017    2580 scope.go:117] "RemoveContainer" containerID="bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4"
Sep 16 13:48:26.607310 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:48:26.607291    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4\": container with ID starting with bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4 not found: ID does not exist" containerID="bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4"
Sep 16 13:48:26.607358 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.607316    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4"} err="failed to get container status \"bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4\": rpc error: code = NotFound desc = could not find container \"bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4\": container with ID starting with bd9fb180c41bf0ecd6fba56f3778426b913eb07ae8cebd328b6e60cb4d6331f4 not found: ID does not exist"
Sep 16 13:48:26.620475 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.620450    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"]
Sep 16 13:48:26.623736 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.623710    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-d94fd9/autores-model-d94fd9-kserve-67f7986dbc-7xdrq"]
Sep 16 13:48:26.802840 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:26.802808    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb2f360-247a-4bb1-8206-a81dcb116772" path="/var/lib/kubelet/pods/4fb2f360-247a-4bb1-8206-a81dcb116772/volumes"
Sep 16 13:48:34.142991 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.142959    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"]
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143277    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d78b22-b565-4f04-ae13-37d98f78fbb5" containerName="payload-pre-processing"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143288    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d78b22-b565-4f04-ae13-37d98f78fbb5" containerName="payload-pre-processing"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143298    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861e51d4-101e-4477-8213-1ec3c41a4b8c" containerName="payload-processing"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143304    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="861e51d4-101e-4477-8213-1ec3c41a4b8c" containerName="payload-processing"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143321    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb2f360-247a-4bb1-8206-a81dcb116772" containerName="main"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143326    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb2f360-247a-4bb1-8206-a81dcb116772" containerName="main"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143332    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91c15950-1cb4-4048-9eb5-2b5728815639" containerName="payload-pre-processing"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143337    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91c15950-1cb4-4048-9eb5-2b5728815639" containerName="payload-pre-processing"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143343    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="572f7d95-e185-4b93-84cb-be257a0bd7e5" containerName="revoke-keys"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143349    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="572f7d95-e185-4b93-84cb-be257a0bd7e5" containerName="revoke-keys"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143362    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53ce719e-ae95-412a-b683-a84ff5c057da" containerName="payload-processing"
Sep 16 13:48:34.143372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143367    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="53ce719e-ae95-412a-b683-a84ff5c057da" containerName="payload-processing"
Sep 16 13:48:34.143760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143412    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d78b22-b565-4f04-ae13-37d98f78fbb5" containerName="payload-pre-processing"
Sep 16 13:48:34.143760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143419    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="572f7d95-e185-4b93-84cb-be257a0bd7e5" containerName="revoke-keys"
Sep 16 13:48:34.143760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143424    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="53ce719e-ae95-412a-b683-a84ff5c057da" containerName="payload-processing"
Sep 16 13:48:34.143760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143431    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="861e51d4-101e-4477-8213-1ec3c41a4b8c" containerName="payload-processing"
Sep 16 13:48:34.143760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143437    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb2f360-247a-4bb1-8206-a81dcb116772" containerName="main"
Sep 16 13:48:34.143760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.143443    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="91c15950-1cb4-4048-9eb5-2b5728815639" containerName="payload-pre-processing"
Sep 16 13:48:34.147724 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.147707    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.150334 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.150315    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-fc8c65\"/\"openshift-service-ca.crt\""
Sep 16 13:48:34.150334 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.150329    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fc8c65\"/\"premium-w4-fc8c65-kserve-self-signed-certs\""
Sep 16 13:48:34.150534 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.150342    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fc8c65\"/\"default-dockercfg-8j9lj\""
Sep 16 13:48:34.150534 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.150380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-fc8c65\"/\"kube-root-ca.crt\""
Sep 16 13:48:34.154297 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.154272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"]
Sep 16 13:48:34.204711 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.204663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8c4444-da71-4439-9604-b55292be7cb1-tls-certs\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.204711 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.204707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scjjm\" (UniqueName: \"kubernetes.io/projected/3c8c4444-da71-4439-9604-b55292be7cb1-kube-api-access-scjjm\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.204887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.204729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-tmp-dir\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.204887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.204809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-model-cache\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.204887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.204843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-dshm\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.204887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.204869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-home\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.305833 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.305798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8c4444-da71-4439-9604-b55292be7cb1-tls-certs\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306013 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.305852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scjjm\" (UniqueName: \"kubernetes.io/projected/3c8c4444-da71-4439-9604-b55292be7cb1-kube-api-access-scjjm\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306013 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.305967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-tmp-dir\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306132 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.306019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-model-cache\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306132 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.306044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-dshm\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306132 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.306067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-home\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306430 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.306386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-tmp-dir\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306430 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.306423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-model-cache\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.306520 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.306502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-home\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.308181 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.308159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-dshm\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.308384 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.308371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8c4444-da71-4439-9604-b55292be7cb1-tls-certs\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.313711 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.313691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scjjm\" (UniqueName: \"kubernetes.io/projected/3c8c4444-da71-4439-9604-b55292be7cb1-kube-api-access-scjjm\") pod \"premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") " pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.457714 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.457638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:34.727160 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:34.727073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"]
Sep 16 13:48:34.730221 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:48:34.730193    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c8c4444_da71_4439_9604_b55292be7cb1.slice/crio-0c682c42be0fae0bf0d19d8fde619ca8e498845cbd79d194f5c82c036aec9fe9 WatchSource:0}: Error finding container 0c682c42be0fae0bf0d19d8fde619ca8e498845cbd79d194f5c82c036aec9fe9: Status 404 returned error can't find the container with id 0c682c42be0fae0bf0d19d8fde619ca8e498845cbd79d194f5c82c036aec9fe9
Sep 16 13:48:35.517336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.517299    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"]
Sep 16 13:48:35.520571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.520554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.522600 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.522575    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-101425\""
Sep 16 13:48:35.531600 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.531574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"]
Sep 16 13:48:35.559887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.559865    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"]
Sep 16 13:48:35.563009 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.562995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:35.564970 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.564947    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-101425-dockercfg-d92kp\""
Sep 16 13:48:35.575001 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.574976    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"]
Sep 16 13:48:35.617492 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.617463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqmnw\" (UniqueName: \"kubernetes.io/projected/5cbe7643-371d-45a7-befe-ce1016523fe7-kube-api-access-nqmnw\") pod \"payload-processing-e2e-explicit-101425-5bcc58d697-626wc\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") " pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:35.617492 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.617494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7fe2881-6883-4145-902a-60ef2510bb0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.617667 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.617519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-698j2\" (UniqueName: \"kubernetes.io/projected/b7fe2881-6883-4145-902a-60ef2510bb0a-kube-api-access-698j2\") pod \"payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.617667 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.617608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cbe7643-371d-45a7-befe-ce1016523fe7-plugins-config-volume\") pod \"payload-processing-e2e-explicit-101425-5bcc58d697-626wc\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") " pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:35.641858 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.641823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw" event={"ID":"3c8c4444-da71-4439-9604-b55292be7cb1","Type":"ContainerStarted","Data":"174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5"}
Sep 16 13:48:35.642010 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.641863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw" event={"ID":"3c8c4444-da71-4439-9604-b55292be7cb1","Type":"ContainerStarted","Data":"0c682c42be0fae0bf0d19d8fde619ca8e498845cbd79d194f5c82c036aec9fe9"}
Sep 16 13:48:35.657193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.657147    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw" podStartSLOduration=1.657131828 podStartE2EDuration="1.657131828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:48:35.655999329 +0000 UTC m=+943.339657725" watchObservedRunningTime="2026-09-16 13:48:35.657131828 +0000 UTC m=+943.340790222"
Sep 16 13:48:35.718617 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.718586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqmnw\" (UniqueName: \"kubernetes.io/projected/5cbe7643-371d-45a7-befe-ce1016523fe7-kube-api-access-nqmnw\") pod \"payload-processing-e2e-explicit-101425-5bcc58d697-626wc\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") " pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:35.718617 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.718615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7fe2881-6883-4145-902a-60ef2510bb0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.718812 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.718766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-698j2\" (UniqueName: \"kubernetes.io/projected/b7fe2881-6883-4145-902a-60ef2510bb0a-kube-api-access-698j2\") pod \"payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.718876 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.718850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cbe7643-371d-45a7-befe-ce1016523fe7-plugins-config-volume\") pod \"payload-processing-e2e-explicit-101425-5bcc58d697-626wc\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") " pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:35.719431 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.719409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7fe2881-6883-4145-902a-60ef2510bb0a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.719584 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.719563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cbe7643-371d-45a7-befe-ce1016523fe7-plugins-config-volume\") pod \"payload-processing-e2e-explicit-101425-5bcc58d697-626wc\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") " pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:35.725704 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.725685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqmnw\" (UniqueName: \"kubernetes.io/projected/5cbe7643-371d-45a7-befe-ce1016523fe7-kube-api-access-nqmnw\") pod \"payload-processing-e2e-explicit-101425-5bcc58d697-626wc\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") " pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:35.725808 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.725789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-698j2\" (UniqueName: \"kubernetes.io/projected/b7fe2881-6883-4145-902a-60ef2510bb0a-kube-api-access-698j2\") pod \"payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.831329 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.831276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:35.872115 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:35.872092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:36.117439 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.117411    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"]
Sep 16 13:48:36.119597 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:48:36.119569    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7fe2881_6883_4145_902a_60ef2510bb0a.slice/crio-78783d6f9898ed7c3bc308bb7f7ac001d7498e195ca14746838ed1c7596aaa24 WatchSource:0}: Error finding container 78783d6f9898ed7c3bc308bb7f7ac001d7498e195ca14746838ed1c7596aaa24: Status 404 returned error can't find the container with id 78783d6f9898ed7c3bc308bb7f7ac001d7498e195ca14746838ed1c7596aaa24
Sep 16 13:48:36.164403 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.164377    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"]
Sep 16 13:48:36.167579 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:48:36.167557    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbe7643_371d_45a7_befe_ce1016523fe7.slice/crio-ca61a052e342d1cd60a8c9582bd3cc914f174be6dbdf7d4f1cf462876af35f3c WatchSource:0}: Error finding container ca61a052e342d1cd60a8c9582bd3cc914f174be6dbdf7d4f1cf462876af35f3c: Status 404 returned error can't find the container with id ca61a052e342d1cd60a8c9582bd3cc914f174be6dbdf7d4f1cf462876af35f3c
Sep 16 13:48:36.648646 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.648609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc" event={"ID":"5cbe7643-371d-45a7-befe-ce1016523fe7","Type":"ContainerStarted","Data":"7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093"}
Sep 16 13:48:36.649211 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.648671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc" event={"ID":"5cbe7643-371d-45a7-befe-ce1016523fe7","Type":"ContainerStarted","Data":"ca61a052e342d1cd60a8c9582bd3cc914f174be6dbdf7d4f1cf462876af35f3c"}
Sep 16 13:48:36.649211 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.648787    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:36.650220 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.650190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9" event={"ID":"b7fe2881-6883-4145-902a-60ef2510bb0a","Type":"ContainerStarted","Data":"1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b"}
Sep 16 13:48:36.650346 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.650237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9" event={"ID":"b7fe2881-6883-4145-902a-60ef2510bb0a","Type":"ContainerStarted","Data":"78783d6f9898ed7c3bc308bb7f7ac001d7498e195ca14746838ed1c7596aaa24"}
Sep 16 13:48:36.650408 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.650387    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:36.666725 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.666664    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc" podStartSLOduration=1.666648451 podStartE2EDuration="1.666648451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:48:36.663731564 +0000 UTC m=+944.347389959" watchObservedRunningTime="2026-09-16 13:48:36.666648451 +0000 UTC m=+944.350306846"
Sep 16 13:48:36.679574 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:36.679537    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9" podStartSLOduration=1.6795240150000001 podStartE2EDuration="1.679524015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:48:36.677882027 +0000 UTC m=+944.361540421" watchObservedRunningTime="2026-09-16 13:48:36.679524015 +0000 UTC m=+944.363182408"
Sep 16 13:48:44.457907 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:44.457862    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:44.458313 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:44.457963    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:44.470436 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:44.470413    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:44.698618 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:44.698591    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:48:47.657051 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:47.657014    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:48:47.657423 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:47.657088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:48:48.838959 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.838925    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"]
Sep 16 13:48:48.847005 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.846981    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:48.849260 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.849237    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7d96c7\"/\"premium-w0-7d96c7-kserve-self-signed-certs\""
Sep 16 13:48:48.849445 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.849427    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"]
Sep 16 13:48:48.850151 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.850103    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7d96c7\"/\"kube-root-ca.crt\""
Sep 16 13:48:48.850151 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.850115    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7d96c7\"/\"openshift-service-ca.crt\""
Sep 16 13:48:48.850315 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.850159    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7d96c7\"/\"default-dockercfg-4k848\""
Sep 16 13:48:48.921219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.921199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-tmp-dir\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:48.921333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.921226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsnk4\" (UniqueName: \"kubernetes.io/projected/53d13dd8-ac40-42de-a40d-d48f581c47b2-kube-api-access-nsnk4\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:48.921333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.921250    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d13dd8-ac40-42de-a40d-d48f581c47b2-tls-certs\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:48.921333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.921268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-home\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:48.921472 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.921345    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-model-cache\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:48.921472 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:48.921394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-dshm\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.022745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.022721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d13dd8-ac40-42de-a40d-d48f581c47b2-tls-certs\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.022851 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.022749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-home\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.022851 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.022775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-model-cache\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.022851 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.022806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-dshm\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.022851 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.022837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-tmp-dir\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.023082 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.022856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsnk4\" (UniqueName: \"kubernetes.io/projected/53d13dd8-ac40-42de-a40d-d48f581c47b2-kube-api-access-nsnk4\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.023158 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.023136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-home\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.023223 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.023184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-model-cache\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.023223 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.023202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-tmp-dir\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.025015 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.024988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-dshm\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.025174 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.025157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d13dd8-ac40-42de-a40d-d48f581c47b2-tls-certs\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.030565 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.030545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsnk4\" (UniqueName: \"kubernetes.io/projected/53d13dd8-ac40-42de-a40d-d48f581c47b2-kube-api-access-nsnk4\") pod \"premium-w0-7d96c7-kserve-5875d69fc8-l8dbb\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") " pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.158020 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.157998    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:49.427332 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.427266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"]
Sep 16 13:48:49.430714 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:48:49.430678    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53d13dd8_ac40_42de_a40d_d48f581c47b2.slice/crio-5d195489d4d3822d0de82de0cdb4e40a3e1d922bc5c4246a82bcccd339eb1cc4 WatchSource:0}: Error finding container 5d195489d4d3822d0de82de0cdb4e40a3e1d922bc5c4246a82bcccd339eb1cc4: Status 404 returned error can't find the container with id 5d195489d4d3822d0de82de0cdb4e40a3e1d922bc5c4246a82bcccd339eb1cc4
Sep 16 13:48:49.713512 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.713426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb" event={"ID":"53d13dd8-ac40-42de-a40d-d48f581c47b2","Type":"ContainerStarted","Data":"d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7"}
Sep 16 13:48:49.713512 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.713468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb" event={"ID":"53d13dd8-ac40-42de-a40d-d48f581c47b2","Type":"ContainerStarted","Data":"5d195489d4d3822d0de82de0cdb4e40a3e1d922bc5c4246a82bcccd339eb1cc4"}
Sep 16 13:48:49.730513 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:49.730467    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb" podStartSLOduration=1.730450867 podStartE2EDuration="1.730450867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:48:49.729563845 +0000 UTC m=+957.413222237" watchObservedRunningTime="2026-09-16 13:48:49.730450867 +0000 UTC m=+957.414109265"
Sep 16 13:48:59.158763 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:59.158723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:59.159173 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:59.158779    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:59.171628 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:59.171605    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:48:59.773116 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:48:59.773086    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:50:07.386914 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.386870    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"]
Sep 16 13:50:07.390963 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.390938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.393158 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.393139    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-51b5e7aa\""
Sep 16 13:50:07.393296 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.393248    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-51b5e7aa-dockercfg-xjxnb\""
Sep 16 13:50:07.400483 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.400459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"]
Sep 16 13:50:07.504437 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.504406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c4067f6-e0fe-4689-b528-5687ad3e703f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.504561 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.504457    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rfdc\" (UniqueName: \"kubernetes.io/projected/5c4067f6-e0fe-4689-b528-5687ad3e703f-kube-api-access-9rfdc\") pod \"payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.604947 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.604912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rfdc\" (UniqueName: \"kubernetes.io/projected/5c4067f6-e0fe-4689-b528-5687ad3e703f-kube-api-access-9rfdc\") pod \"payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.605119 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.605007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c4067f6-e0fe-4689-b528-5687ad3e703f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.605637 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.605618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c4067f6-e0fe-4689-b528-5687ad3e703f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.612996 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.612971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rfdc\" (UniqueName: \"kubernetes.io/projected/5c4067f6-e0fe-4689-b528-5687ad3e703f-kube-api-access-9rfdc\") pod \"payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.702027 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.701951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:07.973856 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:07.973825    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"]
Sep 16 13:50:07.976282 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:50:07.976257    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c4067f6_e0fe_4689_b528_5687ad3e703f.slice/crio-303ecadf8053f0dcf9bac9e89fa1ec32fa9dca69dc362b26be16008197cde430 WatchSource:0}: Error finding container 303ecadf8053f0dcf9bac9e89fa1ec32fa9dca69dc362b26be16008197cde430: Status 404 returned error can't find the container with id 303ecadf8053f0dcf9bac9e89fa1ec32fa9dca69dc362b26be16008197cde430
Sep 16 13:50:08.072491 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:08.072460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj" event={"ID":"5c4067f6-e0fe-4689-b528-5687ad3e703f","Type":"ContainerStarted","Data":"7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a"}
Sep 16 13:50:08.072654 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:08.072509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj" event={"ID":"5c4067f6-e0fe-4689-b528-5687ad3e703f","Type":"ContainerStarted","Data":"303ecadf8053f0dcf9bac9e89fa1ec32fa9dca69dc362b26be16008197cde430"}
Sep 16 13:50:08.072654 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:08.072539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:08.088907 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:08.088861    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj" podStartSLOduration=1.08884689 podStartE2EDuration="1.08884689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:50:08.086884462 +0000 UTC m=+1035.770542853" watchObservedRunningTime="2026-09-16 13:50:08.08884689 +0000 UTC m=+1035.772505307"
Sep 16 13:50:19.079068 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.079037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:19.241454 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.241420    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"]
Sep 16 13:50:19.245072 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.245055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.247163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.247142    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-101425\"/\"kube-root-ca.crt\""
Sep 16 13:50:19.247163 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.247163    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-101425\"/\"default-dockercfg-48f99\""
Sep 16 13:50:19.247346 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.247263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-101425\"/\"explicit-model-101425-kserve-self-signed-certs\""
Sep 16 13:50:19.247415 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.247396    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-101425\"/\"openshift-service-ca.crt\""
Sep 16 13:50:19.252364 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.252244    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"]
Sep 16 13:50:19.303823 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.303794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-dshm\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.303942 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.303834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-model-cache\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.303942 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.303859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a183113-9c5c-4080-a377-9c2f650481e4-tls-certs\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.303942 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.303882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvp96\" (UniqueName: \"kubernetes.io/projected/8a183113-9c5c-4080-a377-9c2f650481e4-kube-api-access-mvp96\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.304046 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.303964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-tmp-dir\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.304046 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.303991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-home\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-model-cache\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a183113-9c5c-4080-a377-9c2f650481e4-tls-certs\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404541 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvp96\" (UniqueName: \"kubernetes.io/projected/8a183113-9c5c-4080-a377-9c2f650481e4-kube-api-access-mvp96\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404541 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-tmp-dir\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404541 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-home\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404541 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-dshm\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404774 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-model-cache\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404882 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-home\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.404965 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.404880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-tmp-dir\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.406648 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.406628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-dshm\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.406829 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.406812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a183113-9c5c-4080-a377-9c2f650481e4-tls-certs\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.411490 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.411470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvp96\" (UniqueName: \"kubernetes.io/projected/8a183113-9c5c-4080-a377-9c2f650481e4-kube-api-access-mvp96\") pod \"explicit-model-101425-kserve-6944bf5f9f-7t6l2\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") " pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.556000 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.555970    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:19.826774 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:19.826749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"]
Sep 16 13:50:19.828578 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:50:19.828552    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a183113_9c5c_4080_a377_9c2f650481e4.slice/crio-1da5a4c79a01d4985f16a5dd6cb119b95b25719f3b73d4462bc1abab1a996e66 WatchSource:0}: Error finding container 1da5a4c79a01d4985f16a5dd6cb119b95b25719f3b73d4462bc1abab1a996e66: Status 404 returned error can't find the container with id 1da5a4c79a01d4985f16a5dd6cb119b95b25719f3b73d4462bc1abab1a996e66
Sep 16 13:50:20.128216 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:20.128188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2" event={"ID":"8a183113-9c5c-4080-a377-9c2f650481e4","Type":"ContainerStarted","Data":"98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e"}
Sep 16 13:50:20.128571 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:20.128227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2" event={"ID":"8a183113-9c5c-4080-a377-9c2f650481e4","Type":"ContainerStarted","Data":"1da5a4c79a01d4985f16a5dd6cb119b95b25719f3b73d4462bc1abab1a996e66"}
Sep 16 13:50:20.145704 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:20.145657    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2" podStartSLOduration=1.145641118 podStartE2EDuration="1.145641118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:50:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:50:20.145140935 +0000 UTC m=+1047.828799330" watchObservedRunningTime="2026-09-16 13:50:20.145641118 +0000 UTC m=+1047.829299566"
Sep 16 13:50:29.556484 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:29.556444    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:29.556860 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:29.556508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:29.569228 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:29.569200    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:30.193447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:30.193361    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:50:40.004870 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.004832    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"]
Sep 16 13:50:40.009062 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.009040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"
Sep 16 13:50:40.011293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.011262    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:50:40.011417 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.011353    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:50:40.014886 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.014862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"]
Sep 16 13:50:40.083038 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.083005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bea7f7-03f8-4ac1-885a-46d2fe360e1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d\" (UID: \"81bea7f7-03f8-4ac1-885a-46d2fe360e1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"
Sep 16 13:50:40.183502 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.183469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bea7f7-03f8-4ac1-885a-46d2fe360e1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d\" (UID: \"81bea7f7-03f8-4ac1-885a-46d2fe360e1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"
Sep 16 13:50:40.184356 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.184332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bea7f7-03f8-4ac1-885a-46d2fe360e1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d\" (UID: \"81bea7f7-03f8-4ac1-885a-46d2fe360e1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"
Sep 16 13:50:40.322730 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.322635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"
Sep 16 13:50:40.604385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:40.604354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"]
Sep 16 13:50:40.607094 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:50:40.607059    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81bea7f7_03f8_4ac1_885a_46d2fe360e1c.slice/crio-44783e9495afef992e276e68002917108042c47a5b381cf7664b55ce52eba226 WatchSource:0}: Error finding container 44783e9495afef992e276e68002917108042c47a5b381cf7664b55ce52eba226: Status 404 returned error can't find the container with id 44783e9495afef992e276e68002917108042c47a5b381cf7664b55ce52eba226
Sep 16 13:50:41.237444 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:41.237356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d" event={"ID":"81bea7f7-03f8-4ac1-885a-46d2fe360e1c","Type":"ContainerStarted","Data":"30886d8df9ff0ac8fc110008e35a58da8bf71e2f425eb49afd8a6a0ebb325a3f"}
Sep 16 13:50:41.237444 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:41.237439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d" event={"ID":"81bea7f7-03f8-4ac1-885a-46d2fe360e1c","Type":"ContainerStarted","Data":"44783e9495afef992e276e68002917108042c47a5b381cf7664b55ce52eba226"}
Sep 16 13:50:41.253997 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:41.253943    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d" podStartSLOduration=2.253928804 podStartE2EDuration="2.253928804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:50:41.252843608 +0000 UTC m=+1068.936502002" watchObservedRunningTime="2026-09-16 13:50:41.253928804 +0000 UTC m=+1068.937587195"
Sep 16 13:50:42.243969 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:42.243930    2580 generic.go:356] "Generic (PLEG): container finished" podID="81bea7f7-03f8-4ac1-885a-46d2fe360e1c" containerID="30886d8df9ff0ac8fc110008e35a58da8bf71e2f425eb49afd8a6a0ebb325a3f" exitCode=0
Sep 16 13:50:42.244365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:42.243980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d" event={"ID":"81bea7f7-03f8-4ac1-885a-46d2fe360e1c","Type":"ContainerDied","Data":"30886d8df9ff0ac8fc110008e35a58da8bf71e2f425eb49afd8a6a0ebb325a3f"}
Sep 16 13:50:43.381796 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:43.381773    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"
Sep 16 13:50:43.515442 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:43.515347    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bea7f7-03f8-4ac1-885a-46d2fe360e1c-maas-api-service-ca\") pod \"81bea7f7-03f8-4ac1-885a-46d2fe360e1c\" (UID: \"81bea7f7-03f8-4ac1-885a-46d2fe360e1c\") "
Sep 16 13:50:43.515779 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:43.515755    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81bea7f7-03f8-4ac1-885a-46d2fe360e1c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81bea7f7-03f8-4ac1-885a-46d2fe360e1c" (UID: "81bea7f7-03f8-4ac1-885a-46d2fe360e1c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:50:43.616846 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:43.616810    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81bea7f7-03f8-4ac1-885a-46d2fe360e1c-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:50:44.255464 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:44.255430    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"
Sep 16 13:50:44.255464 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:44.255456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d" event={"ID":"81bea7f7-03f8-4ac1-885a-46d2fe360e1c","Type":"ContainerDied","Data":"44783e9495afef992e276e68002917108042c47a5b381cf7664b55ce52eba226"}
Sep 16 13:50:44.255719 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:44.255503    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44783e9495afef992e276e68002917108042c47a5b381cf7664b55ce52eba226"
Sep 16 13:50:50.656551 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.656469    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"]
Sep 16 13:50:50.656985 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.656731    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj" podUID="5c4067f6-e0fe-4689-b528-5687ad3e703f" containerName="payload-pre-processing" containerID="cri-o://7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a" gracePeriod=30
Sep 16 13:50:50.810516 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.810481    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"]
Sep 16 13:50:50.813402 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.813372    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-51b5e7aa-ncl4d"]
Sep 16 13:50:50.912400 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.912335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:50.989661 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.989631    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c4067f6-e0fe-4689-b528-5687ad3e703f-plugins-config-volume\") pod \"5c4067f6-e0fe-4689-b528-5687ad3e703f\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") "
Sep 16 13:50:50.989840 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.989686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rfdc\" (UniqueName: \"kubernetes.io/projected/5c4067f6-e0fe-4689-b528-5687ad3e703f-kube-api-access-9rfdc\") pod \"5c4067f6-e0fe-4689-b528-5687ad3e703f\" (UID: \"5c4067f6-e0fe-4689-b528-5687ad3e703f\") "
Sep 16 13:50:50.990095 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.990072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c4067f6-e0fe-4689-b528-5687ad3e703f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c4067f6-e0fe-4689-b528-5687ad3e703f" (UID: "5c4067f6-e0fe-4689-b528-5687ad3e703f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:50:50.991700 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:50.991666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c4067f6-e0fe-4689-b528-5687ad3e703f-kube-api-access-9rfdc" (OuterVolumeSpecName: "kube-api-access-9rfdc") pod "5c4067f6-e0fe-4689-b528-5687ad3e703f" (UID: "5c4067f6-e0fe-4689-b528-5687ad3e703f"). InnerVolumeSpecName "kube-api-access-9rfdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:50:51.090592 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.090558    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c4067f6-e0fe-4689-b528-5687ad3e703f-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:50:51.090592 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.090586    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rfdc\" (UniqueName: \"kubernetes.io/projected/5c4067f6-e0fe-4689-b528-5687ad3e703f-kube-api-access-9rfdc\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:50:51.291999 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.291914    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c4067f6-e0fe-4689-b528-5687ad3e703f" containerID="7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a" exitCode=0
Sep 16 13:50:51.291999 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.291972    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"
Sep 16 13:50:51.292193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.291996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj" event={"ID":"5c4067f6-e0fe-4689-b528-5687ad3e703f","Type":"ContainerDied","Data":"7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a"}
Sep 16 13:50:51.292193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.292050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj" event={"ID":"5c4067f6-e0fe-4689-b528-5687ad3e703f","Type":"ContainerDied","Data":"303ecadf8053f0dcf9bac9e89fa1ec32fa9dca69dc362b26be16008197cde430"}
Sep 16 13:50:51.292193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.292076    2580 scope.go:117] "RemoveContainer" containerID="7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a"
Sep 16 13:50:51.301982 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.301963    2580 scope.go:117] "RemoveContainer" containerID="7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a"
Sep 16 13:50:51.302256 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:50:51.302234    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a\": container with ID starting with 7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a not found: ID does not exist" containerID="7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a"
Sep 16 13:50:51.302314 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.302266    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a"} err="failed to get container status \"7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a\": rpc error: code = NotFound desc = could not find container \"7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a\": container with ID starting with 7924d375f3f53d802271c56d5f68a304d4b58d14ed9d30ffe2d6a5ca5f71307a not found: ID does not exist"
Sep 16 13:50:51.313889 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.313867    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"]
Sep 16 13:50:51.318260 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:51.318239    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-51b5e7aa-6c5d86b546-8fbwj"]
Sep 16 13:50:52.803182 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:52.803146    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c4067f6-e0fe-4689-b528-5687ad3e703f" path="/var/lib/kubelet/pods/5c4067f6-e0fe-4689-b528-5687ad3e703f/volumes"
Sep 16 13:50:52.803513 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:52.803473    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81bea7f7-03f8-4ac1-885a-46d2fe360e1c" path="/var/lib/kubelet/pods/81bea7f7-03f8-4ac1-885a-46d2fe360e1c/volumes"
Sep 16 13:50:55.119869 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.119834    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"]
Sep 16 13:50:55.120234 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.120224    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c4067f6-e0fe-4689-b528-5687ad3e703f" containerName="payload-pre-processing"
Sep 16 13:50:55.120289 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.120235    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c4067f6-e0fe-4689-b528-5687ad3e703f" containerName="payload-pre-processing"
Sep 16 13:50:55.120289 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.120255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81bea7f7-03f8-4ac1-885a-46d2fe360e1c" containerName="revoke-keys"
Sep 16 13:50:55.120289 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.120260    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81bea7f7-03f8-4ac1-885a-46d2fe360e1c" containerName="revoke-keys"
Sep 16 13:50:55.120389 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.120317    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81bea7f7-03f8-4ac1-885a-46d2fe360e1c" containerName="revoke-keys"
Sep 16 13:50:55.120389 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.120328    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c4067f6-e0fe-4689-b528-5687ad3e703f" containerName="payload-pre-processing"
Sep 16 13:50:55.124653 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.124629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"
Sep 16 13:50:55.126509 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.126488    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:50:55.126621 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.126488    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:50:55.129937 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.129910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"]
Sep 16 13:50:55.229177 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.229140    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f68304-e86f-45d0-9e44-673ffe13c6bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-101425-lhhnn\" (UID: \"41f68304-e86f-45d0-9e44-673ffe13c6bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"
Sep 16 13:50:55.330008 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.329961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f68304-e86f-45d0-9e44-673ffe13c6bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-101425-lhhnn\" (UID: \"41f68304-e86f-45d0-9e44-673ffe13c6bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"
Sep 16 13:50:55.330605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.330581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f68304-e86f-45d0-9e44-673ffe13c6bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-101425-lhhnn\" (UID: \"41f68304-e86f-45d0-9e44-673ffe13c6bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"
Sep 16 13:50:55.437849 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.437807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"
Sep 16 13:50:55.712846 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:55.712816    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"]
Sep 16 13:50:55.715522 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:50:55.715488    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f68304_e86f_45d0_9e44_673ffe13c6bc.slice/crio-6029f22cd6082382b133fd3d26d48687e8be33c7f7eb9eadc7fd581c2914ff5f WatchSource:0}: Error finding container 6029f22cd6082382b133fd3d26d48687e8be33c7f7eb9eadc7fd581c2914ff5f: Status 404 returned error can't find the container with id 6029f22cd6082382b133fd3d26d48687e8be33c7f7eb9eadc7fd581c2914ff5f
Sep 16 13:50:56.319066 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:56.318963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn" event={"ID":"41f68304-e86f-45d0-9e44-673ffe13c6bc","Type":"ContainerStarted","Data":"7cd2102babd88016d350150866bb64c9bb02cd8fa387e8ecf4e75fd9a2b6de6c"}
Sep 16 13:50:56.319066 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:56.319021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn" event={"ID":"41f68304-e86f-45d0-9e44-673ffe13c6bc","Type":"ContainerStarted","Data":"6029f22cd6082382b133fd3d26d48687e8be33c7f7eb9eadc7fd581c2914ff5f"}
Sep 16 13:50:56.336859 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:56.336808    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn" podStartSLOduration=1.3367891140000001 podStartE2EDuration="1.336789114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:50:56.336143343 +0000 UTC m=+1084.019801734" watchObservedRunningTime="2026-09-16 13:50:56.336789114 +0000 UTC m=+1084.020447509"
Sep 16 13:50:57.324761 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:57.324720    2580 generic.go:356] "Generic (PLEG): container finished" podID="41f68304-e86f-45d0-9e44-673ffe13c6bc" containerID="7cd2102babd88016d350150866bb64c9bb02cd8fa387e8ecf4e75fd9a2b6de6c" exitCode=0
Sep 16 13:50:57.325168 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:57.324805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn" event={"ID":"41f68304-e86f-45d0-9e44-673ffe13c6bc","Type":"ContainerDied","Data":"7cd2102babd88016d350150866bb64c9bb02cd8fa387e8ecf4e75fd9a2b6de6c"}
Sep 16 13:50:58.471672 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:58.471643    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"
Sep 16 13:50:58.558978 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:58.558942    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f68304-e86f-45d0-9e44-673ffe13c6bc-maas-api-service-ca\") pod \"41f68304-e86f-45d0-9e44-673ffe13c6bc\" (UID: \"41f68304-e86f-45d0-9e44-673ffe13c6bc\") "
Sep 16 13:50:58.559354 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:58.559329    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f68304-e86f-45d0-9e44-673ffe13c6bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41f68304-e86f-45d0-9e44-673ffe13c6bc" (UID: "41f68304-e86f-45d0-9e44-673ffe13c6bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:50:58.660329 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:58.660289    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f68304-e86f-45d0-9e44-673ffe13c6bc-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:50:59.336226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:59.336182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn" event={"ID":"41f68304-e86f-45d0-9e44-673ffe13c6bc","Type":"ContainerDied","Data":"6029f22cd6082382b133fd3d26d48687e8be33c7f7eb9eadc7fd581c2914ff5f"}
Sep 16 13:50:59.336226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:59.336215    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"
Sep 16 13:50:59.336226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:50:59.336237    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6029f22cd6082382b133fd3d26d48687e8be33c7f7eb9eadc7fd581c2914ff5f"
Sep 16 13:51:02.545702 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.545663    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"]
Sep 16 13:51:02.546288 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.546269    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41f68304-e86f-45d0-9e44-673ffe13c6bc" containerName="revoke-keys"
Sep 16 13:51:02.546341 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.546292    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41f68304-e86f-45d0-9e44-673ffe13c6bc" containerName="revoke-keys"
Sep 16 13:51:02.546406 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.546395    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="41f68304-e86f-45d0-9e44-673ffe13c6bc" containerName="revoke-keys"
Sep 16 13:51:02.549455 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.549430    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.551519 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.551496    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7d96c7\"/\"distinct-w0-7d96c7-kserve-self-signed-certs\""
Sep 16 13:51:02.558458 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.558434    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"]
Sep 16 13:51:02.695316 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.695274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-dshm\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.695518 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.695330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-home\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.695518 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.695362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tls-certs\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.695518 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.695416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p5vv\" (UniqueName: \"kubernetes.io/projected/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-kube-api-access-8p5vv\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.695518 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.695480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-model-cache\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.695679 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.695518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tmp-dir\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.796532 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.796424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p5vv\" (UniqueName: \"kubernetes.io/projected/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-kube-api-access-8p5vv\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.796532 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.796488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-model-cache\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.796532 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.796521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tmp-dir\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.796833 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.796574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-dshm\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.796833 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.796604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-home\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.796833 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.796628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tls-certs\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.797040 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.796997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-model-cache\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.797101 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.797064    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-home\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.797185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.797149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tmp-dir\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.799082 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.799047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-dshm\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.799276 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.799255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tls-certs\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.805173 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.805149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p5vv\" (UniqueName: \"kubernetes.io/projected/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-kube-api-access-8p5vv\") pod \"distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") " pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:02.860310 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:02.860264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:03.132186 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:03.132161    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"]
Sep 16 13:51:03.134432 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:51:03.134405    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84128b40_1e4d_43c7_ad8f_fc3691f2fb63.slice/crio-111790159dca58e7704d229a3604b332eec1d38a433f7277bb287a3c0fc703d1 WatchSource:0}: Error finding container 111790159dca58e7704d229a3604b332eec1d38a433f7277bb287a3c0fc703d1: Status 404 returned error can't find the container with id 111790159dca58e7704d229a3604b332eec1d38a433f7277bb287a3c0fc703d1
Sep 16 13:51:03.356567 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:03.356530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll" event={"ID":"84128b40-1e4d-43c7-ad8f-fc3691f2fb63","Type":"ContainerStarted","Data":"db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7"}
Sep 16 13:51:03.356567 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:03.356572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll" event={"ID":"84128b40-1e4d-43c7-ad8f-fc3691f2fb63","Type":"ContainerStarted","Data":"111790159dca58e7704d229a3604b332eec1d38a433f7277bb287a3c0fc703d1"}
Sep 16 13:51:03.371748 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:03.371699    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll" podStartSLOduration=1.371684058 podStartE2EDuration="1.371684058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:51:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:51:03.370638083 +0000 UTC m=+1091.054296476" watchObservedRunningTime="2026-09-16 13:51:03.371684058 +0000 UTC m=+1091.055342452"
Sep 16 13:51:05.433045 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:05.433010    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"]
Sep 16 13:51:06.014331 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.014295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"]
Sep 16 13:51:06.014545 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.014517    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc" podUID="5cbe7643-371d-45a7-befe-ce1016523fe7" containerName="payload-processing" containerID="cri-o://7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093" gracePeriod=30
Sep 16 13:51:06.028149 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.027824    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"]
Sep 16 13:51:06.028286 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.028196    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9" podUID="b7fe2881-6883-4145-902a-60ef2510bb0a" containerName="payload-pre-processing" containerID="cri-o://1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b" gracePeriod=30
Sep 16 13:51:06.187702 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.187646    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"]
Sep 16 13:51:06.189698 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.189667    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-101425-lhhnn"]
Sep 16 13:51:06.291131 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.291098    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:51:06.317247 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.317221    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:51:06.372281 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.372245    2580 generic.go:356] "Generic (PLEG): container finished" podID="5cbe7643-371d-45a7-befe-ce1016523fe7" containerID="7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093" exitCode=0
Sep 16 13:51:06.372453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.372292    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"
Sep 16 13:51:06.372453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.372338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc" event={"ID":"5cbe7643-371d-45a7-befe-ce1016523fe7","Type":"ContainerDied","Data":"7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093"}
Sep 16 13:51:06.372453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.372393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc" event={"ID":"5cbe7643-371d-45a7-befe-ce1016523fe7","Type":"ContainerDied","Data":"ca61a052e342d1cd60a8c9582bd3cc914f174be6dbdf7d4f1cf462876af35f3c"}
Sep 16 13:51:06.372453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.372421    2580 scope.go:117] "RemoveContainer" containerID="7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093"
Sep 16 13:51:06.373865 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.373831    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7fe2881-6883-4145-902a-60ef2510bb0a" containerID="1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b" exitCode=0
Sep 16 13:51:06.374011 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.373876    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"
Sep 16 13:51:06.374011 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.373892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9" event={"ID":"b7fe2881-6883-4145-902a-60ef2510bb0a","Type":"ContainerDied","Data":"1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b"}
Sep 16 13:51:06.374011 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.373945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9" event={"ID":"b7fe2881-6883-4145-902a-60ef2510bb0a","Type":"ContainerDied","Data":"78783d6f9898ed7c3bc308bb7f7ac001d7498e195ca14746838ed1c7596aaa24"}
Sep 16 13:51:06.382052 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.382031    2580 scope.go:117] "RemoveContainer" containerID="7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093"
Sep 16 13:51:06.382325 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:51:06.382302    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093\": container with ID starting with 7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093 not found: ID does not exist" containerID="7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093"
Sep 16 13:51:06.382397 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.382333    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093"} err="failed to get container status \"7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093\": rpc error: code = NotFound desc = could not find container \"7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093\": container with ID starting with 7cab7233585cfb9c6a33c05196e68d76fcd6c9444215e2d728fd75154ea1d093 not found: ID does not exist"
Sep 16 13:51:06.382397 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.382349    2580 scope.go:117] "RemoveContainer" containerID="1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b"
Sep 16 13:51:06.389938 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.389916    2580 scope.go:117] "RemoveContainer" containerID="1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b"
Sep 16 13:51:06.390240 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:51:06.390221    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b\": container with ID starting with 1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b not found: ID does not exist" containerID="1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b"
Sep 16 13:51:06.390290 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.390246    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b"} err="failed to get container status \"1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b\": rpc error: code = NotFound desc = could not find container \"1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b\": container with ID starting with 1c8f0ff24b38b840301238f43ac374698bc09eb7e8bc2ff60d88be507ce3344b not found: ID does not exist"
Sep 16 13:51:06.428400 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.428360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-698j2\" (UniqueName: \"kubernetes.io/projected/b7fe2881-6883-4145-902a-60ef2510bb0a-kube-api-access-698j2\") pod \"b7fe2881-6883-4145-902a-60ef2510bb0a\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") "
Sep 16 13:51:06.428400 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.428402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqmnw\" (UniqueName: \"kubernetes.io/projected/5cbe7643-371d-45a7-befe-ce1016523fe7-kube-api-access-nqmnw\") pod \"5cbe7643-371d-45a7-befe-ce1016523fe7\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") "
Sep 16 13:51:06.428640 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.428442    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7fe2881-6883-4145-902a-60ef2510bb0a-plugins-config-volume\") pod \"b7fe2881-6883-4145-902a-60ef2510bb0a\" (UID: \"b7fe2881-6883-4145-902a-60ef2510bb0a\") "
Sep 16 13:51:06.428640 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.428493    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cbe7643-371d-45a7-befe-ce1016523fe7-plugins-config-volume\") pod \"5cbe7643-371d-45a7-befe-ce1016523fe7\" (UID: \"5cbe7643-371d-45a7-befe-ce1016523fe7\") "
Sep 16 13:51:06.428827 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.428800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7fe2881-6883-4145-902a-60ef2510bb0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7fe2881-6883-4145-902a-60ef2510bb0a" (UID: "b7fe2881-6883-4145-902a-60ef2510bb0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:51:06.428987 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.428872    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cbe7643-371d-45a7-befe-ce1016523fe7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5cbe7643-371d-45a7-befe-ce1016523fe7" (UID: "5cbe7643-371d-45a7-befe-ce1016523fe7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:51:06.430541 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.430521    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7fe2881-6883-4145-902a-60ef2510bb0a-kube-api-access-698j2" (OuterVolumeSpecName: "kube-api-access-698j2") pod "b7fe2881-6883-4145-902a-60ef2510bb0a" (UID: "b7fe2881-6883-4145-902a-60ef2510bb0a"). InnerVolumeSpecName "kube-api-access-698j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:51:06.430648 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.430544    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cbe7643-371d-45a7-befe-ce1016523fe7-kube-api-access-nqmnw" (OuterVolumeSpecName: "kube-api-access-nqmnw") pod "5cbe7643-371d-45a7-befe-ce1016523fe7" (UID: "5cbe7643-371d-45a7-befe-ce1016523fe7"). InnerVolumeSpecName "kube-api-access-nqmnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:51:06.529209 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.529124    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-698j2\" (UniqueName: \"kubernetes.io/projected/b7fe2881-6883-4145-902a-60ef2510bb0a-kube-api-access-698j2\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:06.529209 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.529153    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqmnw\" (UniqueName: \"kubernetes.io/projected/5cbe7643-371d-45a7-befe-ce1016523fe7-kube-api-access-nqmnw\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:06.529209 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.529163    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7fe2881-6883-4145-902a-60ef2510bb0a-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:06.529209 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.529173    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cbe7643-371d-45a7-befe-ce1016523fe7-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:06.693745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.693715    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"]
Sep 16 13:51:06.700737 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.700708    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-101425-5bcc58d697-626wc"]
Sep 16 13:51:06.709266 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.709241    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"]
Sep 16 13:51:06.719972 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.719943    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-101425-8d55cd74-nz5s9"]
Sep 16 13:51:06.802459 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.802372    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41f68304-e86f-45d0-9e44-673ffe13c6bc" path="/var/lib/kubelet/pods/41f68304-e86f-45d0-9e44-673ffe13c6bc/volumes"
Sep 16 13:51:06.802804 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.802789    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cbe7643-371d-45a7-befe-ce1016523fe7" path="/var/lib/kubelet/pods/5cbe7643-371d-45a7-befe-ce1016523fe7/volumes"
Sep 16 13:51:06.803170 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:06.803157    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7fe2881-6883-4145-902a-60ef2510bb0a" path="/var/lib/kubelet/pods/b7fe2881-6883-4145-902a-60ef2510bb0a/volumes"
Sep 16 13:51:12.861389 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:12.861354    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:12.861894 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:12.861431    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:12.874455 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:12.874429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:13.420235 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:13.420198    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:51:18.541000 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.540965    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"]
Sep 16 13:51:18.541393 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.541298    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7fe2881-6883-4145-902a-60ef2510bb0a" containerName="payload-pre-processing"
Sep 16 13:51:18.541393 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.541309    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7fe2881-6883-4145-902a-60ef2510bb0a" containerName="payload-pre-processing"
Sep 16 13:51:18.541393 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.541326    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbe7643-371d-45a7-befe-ce1016523fe7" containerName="payload-processing"
Sep 16 13:51:18.541393 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.541333    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbe7643-371d-45a7-befe-ce1016523fe7" containerName="payload-processing"
Sep 16 13:51:18.541543 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.541398    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7fe2881-6883-4145-902a-60ef2510bb0a" containerName="payload-pre-processing"
Sep 16 13:51:18.541543 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.541409    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cbe7643-371d-45a7-befe-ce1016523fe7" containerName="payload-processing"
Sep 16 13:51:18.545685 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.545663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:18.547697 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.547672    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-4fa336-dockercfg-t5wn2\""
Sep 16 13:51:18.547814 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.547730    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-4fa336\""
Sep 16 13:51:18.561824 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.561789    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"]
Sep 16 13:51:18.626460 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.626421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:18.626618 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.626476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j57bt\" (UniqueName: \"kubernetes.io/projected/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-kube-api-access-j57bt\") pod \"payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:18.727061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.727022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:18.727247 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.727080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j57bt\" (UniqueName: \"kubernetes.io/projected/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-kube-api-access-j57bt\") pod \"payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:18.727788 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.727766    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:18.734370 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.734345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j57bt\" (UniqueName: \"kubernetes.io/projected/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-kube-api-access-j57bt\") pod \"payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:18.858008 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:18.857895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:19.133753 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:19.133728    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"]
Sep 16 13:51:19.136402 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:51:19.136372    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb632d5_4bf5_4af9_a7eb_26c6424a3750.slice/crio-de2bcce3bf5768267e8e390f39e21728153e651fa8ce2b8a4ffd8ce8f9e62f27 WatchSource:0}: Error finding container de2bcce3bf5768267e8e390f39e21728153e651fa8ce2b8a4ffd8ce8f9e62f27: Status 404 returned error can't find the container with id de2bcce3bf5768267e8e390f39e21728153e651fa8ce2b8a4ffd8ce8f9e62f27
Sep 16 13:51:19.440122 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:19.440022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl" event={"ID":"4cb632d5-4bf5-4af9-a7eb-26c6424a3750","Type":"ContainerStarted","Data":"09ed1d216fb972603c71745d0af4894041fb5b5eab202e37f6a3e7cb8765896b"}
Sep 16 13:51:19.440122 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:19.440075    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl" event={"ID":"4cb632d5-4bf5-4af9-a7eb-26c6424a3750","Type":"ContainerStarted","Data":"de2bcce3bf5768267e8e390f39e21728153e651fa8ce2b8a4ffd8ce8f9e62f27"}
Sep 16 13:51:19.440385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:19.440142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:19.455958 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:19.455882    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl" podStartSLOduration=1.455868221 podStartE2EDuration="1.455868221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:51:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:51:19.454295536 +0000 UTC m=+1107.137953930" watchObservedRunningTime="2026-09-16 13:51:19.455868221 +0000 UTC m=+1107.139526615"
Sep 16 13:51:20.452607 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.452567    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2" podUID="8a183113-9c5c-4080-a377-9c2f650481e4" containerName="main" containerID="cri-o://98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e" gracePeriod=45
Sep 16 13:51:20.714980 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.714921    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:51:20.746515 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746489    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-dshm\") pod \"8a183113-9c5c-4080-a377-9c2f650481e4\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") "
Sep 16 13:51:20.746668 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvp96\" (UniqueName: \"kubernetes.io/projected/8a183113-9c5c-4080-a377-9c2f650481e4-kube-api-access-mvp96\") pod \"8a183113-9c5c-4080-a377-9c2f650481e4\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") "
Sep 16 13:51:20.746668 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746568    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-model-cache\") pod \"8a183113-9c5c-4080-a377-9c2f650481e4\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") "
Sep 16 13:51:20.746668 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746610    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-home\") pod \"8a183113-9c5c-4080-a377-9c2f650481e4\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") "
Sep 16 13:51:20.746668 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-tmp-dir\") pod \"8a183113-9c5c-4080-a377-9c2f650481e4\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") "
Sep 16 13:51:20.746924 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746702    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a183113-9c5c-4080-a377-9c2f650481e4-tls-certs\") pod \"8a183113-9c5c-4080-a377-9c2f650481e4\" (UID: \"8a183113-9c5c-4080-a377-9c2f650481e4\") "
Sep 16 13:51:20.746924 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a183113-9c5c-4080-a377-9c2f650481e4" (UID: "8a183113-9c5c-4080-a377-9c2f650481e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:51:20.747050 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746925    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-home" (OuterVolumeSpecName: "home") pod "8a183113-9c5c-4080-a377-9c2f650481e4" (UID: "8a183113-9c5c-4080-a377-9c2f650481e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:51:20.747050 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.746952    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a183113-9c5c-4080-a377-9c2f650481e4" (UID: "8a183113-9c5c-4080-a377-9c2f650481e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:51:20.747148 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.747056    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:20.747148 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.747075    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:20.747148 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.747088    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:20.748790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.748762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-dshm" (OuterVolumeSpecName: "dshm") pod "8a183113-9c5c-4080-a377-9c2f650481e4" (UID: "8a183113-9c5c-4080-a377-9c2f650481e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:51:20.748887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.748847    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a183113-9c5c-4080-a377-9c2f650481e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a183113-9c5c-4080-a377-9c2f650481e4" (UID: "8a183113-9c5c-4080-a377-9c2f650481e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:51:20.748948 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.748881    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a183113-9c5c-4080-a377-9c2f650481e4-kube-api-access-mvp96" (OuterVolumeSpecName: "kube-api-access-mvp96") pod "8a183113-9c5c-4080-a377-9c2f650481e4" (UID: "8a183113-9c5c-4080-a377-9c2f650481e4"). InnerVolumeSpecName "kube-api-access-mvp96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:51:20.848031 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.847993    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a183113-9c5c-4080-a377-9c2f650481e4-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:20.848031 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.848023    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a183113-9c5c-4080-a377-9c2f650481e4-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:20.848031 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:20.848034    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvp96\" (UniqueName: \"kubernetes.io/projected/8a183113-9c5c-4080-a377-9c2f650481e4-kube-api-access-mvp96\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:21.450669 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.450632    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a183113-9c5c-4080-a377-9c2f650481e4" containerID="98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e" exitCode=0
Sep 16 13:51:21.450858 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.450679    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"
Sep 16 13:51:21.450858 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.450689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2" event={"ID":"8a183113-9c5c-4080-a377-9c2f650481e4","Type":"ContainerDied","Data":"98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e"}
Sep 16 13:51:21.450858 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.450732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2" event={"ID":"8a183113-9c5c-4080-a377-9c2f650481e4","Type":"ContainerDied","Data":"1da5a4c79a01d4985f16a5dd6cb119b95b25719f3b73d4462bc1abab1a996e66"}
Sep 16 13:51:21.450858 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.450748    2580 scope.go:117] "RemoveContainer" containerID="98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e"
Sep 16 13:51:21.459730 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.459533    2580 scope.go:117] "RemoveContainer" containerID="98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e"
Sep 16 13:51:21.460028 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:51:21.459838    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e\": container with ID starting with 98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e not found: ID does not exist" containerID="98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e"
Sep 16 13:51:21.460028 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.459864    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e"} err="failed to get container status \"98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e\": rpc error: code = NotFound desc = could not find container \"98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e\": container with ID starting with 98196e5c9edb80b7299148f90c64406eec497dee047fed8fd089b9e2a05d860e not found: ID does not exist"
Sep 16 13:51:21.466110 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.466086    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"]
Sep 16 13:51:21.469794 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:21.469760    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-101425/explicit-model-101425-kserve-6944bf5f9f-7t6l2"]
Sep 16 13:51:22.805421 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:22.805383    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a183113-9c5c-4080-a377-9c2f650481e4" path="/var/lib/kubelet/pods/8a183113-9c5c-4080-a377-9c2f650481e4/volumes"
Sep 16 13:51:30.447350 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:30.447321    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:51:35.592841 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.592803    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"]
Sep 16 13:51:35.593372 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.593349    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a183113-9c5c-4080-a377-9c2f650481e4" containerName="main"
Sep 16 13:51:35.593456 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.593377    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a183113-9c5c-4080-a377-9c2f650481e4" containerName="main"
Sep 16 13:51:35.600178 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.600150    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a183113-9c5c-4080-a377-9c2f650481e4" containerName="main"
Sep 16 13:51:35.607161 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.607116    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"]
Sep 16 13:51:35.607288 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.607228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"
Sep 16 13:51:35.609329 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.609307    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:51:35.609470 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.609357    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:51:35.672431 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.672400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43765abe-8c30-4658-b4c4-bf423daae560-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-401a67-t97vr\" (UID: \"43765abe-8c30-4658-b4c4-bf423daae560\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"
Sep 16 13:51:35.773852 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.773822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43765abe-8c30-4658-b4c4-bf423daae560-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-401a67-t97vr\" (UID: \"43765abe-8c30-4658-b4c4-bf423daae560\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"
Sep 16 13:51:35.774409 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.774389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43765abe-8c30-4658-b4c4-bf423daae560-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-401a67-t97vr\" (UID: \"43765abe-8c30-4658-b4c4-bf423daae560\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"
Sep 16 13:51:35.920874 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:35.920845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"
Sep 16 13:51:36.190191 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:36.190166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"]
Sep 16 13:51:36.192566 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:51:36.192528    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43765abe_8c30_4658_b4c4_bf423daae560.slice/crio-38ce0af727cd4da78d660a7f8c6424ca617398c462723381090936fb5f6efa13 WatchSource:0}: Error finding container 38ce0af727cd4da78d660a7f8c6424ca617398c462723381090936fb5f6efa13: Status 404 returned error can't find the container with id 38ce0af727cd4da78d660a7f8c6424ca617398c462723381090936fb5f6efa13
Sep 16 13:51:36.532026 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:36.531990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr" event={"ID":"43765abe-8c30-4658-b4c4-bf423daae560","Type":"ContainerStarted","Data":"b889c743574048c13d4e80534adbdd658fb237dd8f312afeb08a1f28c806e957"}
Sep 16 13:51:36.532133 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:36.532047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr" event={"ID":"43765abe-8c30-4658-b4c4-bf423daae560","Type":"ContainerStarted","Data":"38ce0af727cd4da78d660a7f8c6424ca617398c462723381090936fb5f6efa13"}
Sep 16 13:51:36.546718 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:36.546668    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr" podStartSLOduration=1.546653506 podStartE2EDuration="1.546653506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:51:36.545719112 +0000 UTC m=+1124.229377518" watchObservedRunningTime="2026-09-16 13:51:36.546653506 +0000 UTC m=+1124.230311900"
Sep 16 13:51:37.538392 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:37.538357    2580 generic.go:356] "Generic (PLEG): container finished" podID="43765abe-8c30-4658-b4c4-bf423daae560" containerID="b889c743574048c13d4e80534adbdd658fb237dd8f312afeb08a1f28c806e957" exitCode=0
Sep 16 13:51:37.538785 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:37.538408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr" event={"ID":"43765abe-8c30-4658-b4c4-bf423daae560","Type":"ContainerDied","Data":"b889c743574048c13d4e80534adbdd658fb237dd8f312afeb08a1f28c806e957"}
Sep 16 13:51:38.680810 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:38.680785    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"
Sep 16 13:51:38.800548 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:38.800457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43765abe-8c30-4658-b4c4-bf423daae560-maas-api-service-ca\") pod \"43765abe-8c30-4658-b4c4-bf423daae560\" (UID: \"43765abe-8c30-4658-b4c4-bf423daae560\") "
Sep 16 13:51:38.800817 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:38.800792    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43765abe-8c30-4658-b4c4-bf423daae560-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "43765abe-8c30-4658-b4c4-bf423daae560" (UID: "43765abe-8c30-4658-b4c4-bf423daae560"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:51:38.901420 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:38.901387    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43765abe-8c30-4658-b4c4-bf423daae560-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:39.549664 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:39.549635    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"
Sep 16 13:51:39.549832 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:39.549626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr" event={"ID":"43765abe-8c30-4658-b4c4-bf423daae560","Type":"ContainerDied","Data":"38ce0af727cd4da78d660a7f8c6424ca617398c462723381090936fb5f6efa13"}
Sep 16 13:51:39.549832 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:39.549768    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38ce0af727cd4da78d660a7f8c6424ca617398c462723381090936fb5f6efa13"
Sep 16 13:51:46.273199 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.273164    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"]
Sep 16 13:51:46.273574 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.273503    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43765abe-8c30-4658-b4c4-bf423daae560" containerName="revoke-keys"
Sep 16 13:51:46.273574 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.273514    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43765abe-8c30-4658-b4c4-bf423daae560" containerName="revoke-keys"
Sep 16 13:51:46.273648 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.273579    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="43765abe-8c30-4658-b4c4-bf423daae560" containerName="revoke-keys"
Sep 16 13:51:46.276531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.276512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.278565 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.278532    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-284b60\""
Sep 16 13:51:46.278727 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.278709    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-284b60-dockercfg-6kg7p\""
Sep 16 13:51:46.286923 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.286883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"]
Sep 16 13:51:46.320878 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.320853    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"]
Sep 16 13:51:46.324059 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.324044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.334887 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.334866    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"]
Sep 16 13:51:46.369765 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.369731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea97e996-a2fe-4c6b-81c1-766645456f5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.369891 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.369785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzh42\" (UniqueName: \"kubernetes.io/projected/ea97e996-a2fe-4c6b-81c1-766645456f5c-kube-api-access-fzh42\") pod \"payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.470790 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.470753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzh42\" (UniqueName: \"kubernetes.io/projected/ea97e996-a2fe-4c6b-81c1-766645456f5c-kube-api-access-fzh42\") pod \"payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.470981 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.470809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef73f2a4-84ec-4b34-acca-133473411a1a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.470981 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.470848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbhpk\" (UniqueName: \"kubernetes.io/projected/ef73f2a4-84ec-4b34-acca-133473411a1a-kube-api-access-wbhpk\") pod \"payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.470981 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.470870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea97e996-a2fe-4c6b-81c1-766645456f5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.471523 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.471501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea97e996-a2fe-4c6b-81c1-766645456f5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.478708 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.478673    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzh42\" (UniqueName: \"kubernetes.io/projected/ea97e996-a2fe-4c6b-81c1-766645456f5c-kube-api-access-fzh42\") pod \"payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.500079 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.500046    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"]
Sep 16 13:51:46.500737 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.500704    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2" podUID="0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" containerName="payload-pre-processing" containerID="cri-o://9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302" gracePeriod=30
Sep 16 13:51:46.571648 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.571605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbhpk\" (UniqueName: \"kubernetes.io/projected/ef73f2a4-84ec-4b34-acca-133473411a1a-kube-api-access-wbhpk\") pod \"payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.571782 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.571708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef73f2a4-84ec-4b34-acca-133473411a1a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.572404 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.572384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef73f2a4-84ec-4b34-acca-133473411a1a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.582111 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.582085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbhpk\" (UniqueName: \"kubernetes.io/projected/ef73f2a4-84ec-4b34-acca-133473411a1a-kube-api-access-wbhpk\") pod \"payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.587183 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.587163    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:46.633886 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.633852    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:46.741744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.741686    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"]
Sep 16 13:51:46.744591 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.744558    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-401a67-t97vr"]
Sep 16 13:51:46.779632 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.779546    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:51:46.805061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.805028    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43765abe-8c30-4658-b4c4-bf423daae560" path="/var/lib/kubelet/pods/43765abe-8c30-4658-b4c4-bf423daae560/volumes"
Sep 16 13:51:46.864973 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.864950    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"]
Sep 16 13:51:46.867697 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:51:46.867660    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea97e996_a2fe_4c6b_81c1_766645456f5c.slice/crio-1a99288a54c95a4d3975c9c5d3e26eeadded1e94a88099dd2646c78f57c5025f WatchSource:0}: Error finding container 1a99288a54c95a4d3975c9c5d3e26eeadded1e94a88099dd2646c78f57c5025f: Status 404 returned error can't find the container with id 1a99288a54c95a4d3975c9c5d3e26eeadded1e94a88099dd2646c78f57c5025f
Sep 16 13:51:46.877108 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.877085    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-plugins-config-volume\") pod \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") "
Sep 16 13:51:46.877246 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.877195    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29xq8\" (UniqueName: \"kubernetes.io/projected/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-kube-api-access-29xq8\") pod \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\" (UID: \"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5\") "
Sep 16 13:51:46.877507 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.877482    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" (UID: "0f8d7119-4f57-46d3-8685-a7d8b12fc7c5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:51:46.879170 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.879147    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-kube-api-access-29xq8" (OuterVolumeSpecName: "kube-api-access-29xq8") pod "0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" (UID: "0f8d7119-4f57-46d3-8685-a7d8b12fc7c5"). InnerVolumeSpecName "kube-api-access-29xq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:51:46.924745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.924715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"]
Sep 16 13:51:46.927304 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:51:46.927280    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef73f2a4_84ec_4b34_acca_133473411a1a.slice/crio-bd74fc14016198f1032bc819893522654451317db04e26a2fb39fd587b9ede1f WatchSource:0}: Error finding container bd74fc14016198f1032bc819893522654451317db04e26a2fb39fd587b9ede1f: Status 404 returned error can't find the container with id bd74fc14016198f1032bc819893522654451317db04e26a2fb39fd587b9ede1f
Sep 16 13:51:46.977927 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.977881    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:46.977927 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:46.977925    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29xq8\" (UniqueName: \"kubernetes.io/projected/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5-kube-api-access-29xq8\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:47.595068 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.595033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj" event={"ID":"ef73f2a4-84ec-4b34-acca-133473411a1a","Type":"ContainerStarted","Data":"e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77"}
Sep 16 13:51:47.595530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.595223    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:51:47.595530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.595259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj" event={"ID":"ef73f2a4-84ec-4b34-acca-133473411a1a","Type":"ContainerStarted","Data":"bd74fc14016198f1032bc819893522654451317db04e26a2fb39fd587b9ede1f"}
Sep 16 13:51:47.596397 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.596374    2580 generic.go:356] "Generic (PLEG): container finished" podID="0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" containerID="9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302" exitCode=0
Sep 16 13:51:47.596475 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.596413    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"
Sep 16 13:51:47.596475 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.596423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2" event={"ID":"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5","Type":"ContainerDied","Data":"9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302"}
Sep 16 13:51:47.596475 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.596465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2" event={"ID":"0f8d7119-4f57-46d3-8685-a7d8b12fc7c5","Type":"ContainerDied","Data":"566191d5b36693aa9d2a8cee0d1be02973224ee9bab21a0b768a32c93108efbb"}
Sep 16 13:51:47.596626 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.596488    2580 scope.go:117] "RemoveContainer" containerID="9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302"
Sep 16 13:51:47.598376 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.598348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv" event={"ID":"ea97e996-a2fe-4c6b-81c1-766645456f5c","Type":"ContainerStarted","Data":"c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c"}
Sep 16 13:51:47.598489 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.598398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv" event={"ID":"ea97e996-a2fe-4c6b-81c1-766645456f5c","Type":"ContainerStarted","Data":"1a99288a54c95a4d3975c9c5d3e26eeadded1e94a88099dd2646c78f57c5025f"}
Sep 16 13:51:47.598489 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.598428    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:47.606062 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.606031    2580 scope.go:117] "RemoveContainer" containerID="9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302"
Sep 16 13:51:47.606333 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:51:47.606315    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302\": container with ID starting with 9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302 not found: ID does not exist" containerID="9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302"
Sep 16 13:51:47.606384 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.606342    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302"} err="failed to get container status \"9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302\": rpc error: code = NotFound desc = could not find container \"9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302\": container with ID starting with 9b5b796c737e62f31a19f107be9abfb6cc971b72178b9b5baad29cf69189a302 not found: ID does not exist"
Sep 16 13:51:47.612344 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.612298    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj" podStartSLOduration=1.6122826479999999 podStartE2EDuration="1.612282648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:51:47.610492183 +0000 UTC m=+1135.294150588" watchObservedRunningTime="2026-09-16 13:51:47.612282648 +0000 UTC m=+1135.295941045"
Sep 16 13:51:47.629638 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.629598    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv" podStartSLOduration=1.629585691 podStartE2EDuration="1.629585691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:51:47.62782496 +0000 UTC m=+1135.311483353" watchObservedRunningTime="2026-09-16 13:51:47.629585691 +0000 UTC m=+1135.313244085"
Sep 16 13:51:47.642076 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.642054    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"]
Sep 16 13:51:47.643611 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:47.643593    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-401a67-69549bb69d-6z6f2"]
Sep 16 13:51:48.804275 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:48.804241    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" path="/var/lib/kubelet/pods/0f8d7119-4f57-46d3-8685-a7d8b12fc7c5/volumes"
Sep 16 13:51:53.577996 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.577956    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"]
Sep 16 13:51:53.578388 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.578334    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" containerName="payload-pre-processing"
Sep 16 13:51:53.578388 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.578347    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" containerName="payload-pre-processing"
Sep 16 13:51:53.578469 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.578402    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f8d7119-4f57-46d3-8685-a7d8b12fc7c5" containerName="payload-pre-processing"
Sep 16 13:51:53.582620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.582602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"
Sep 16 13:51:53.584586 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.584567    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:51:53.584691 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.584567    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:51:53.587620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.587601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"]
Sep 16 13:51:53.740345 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.740310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1527ada-72ef-4120-bc7c-22aaea22dc8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw\" (UID: \"d1527ada-72ef-4120-bc7c-22aaea22dc8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"
Sep 16 13:51:53.840926 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.840825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1527ada-72ef-4120-bc7c-22aaea22dc8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw\" (UID: \"d1527ada-72ef-4120-bc7c-22aaea22dc8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"
Sep 16 13:51:53.841487 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.841466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1527ada-72ef-4120-bc7c-22aaea22dc8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw\" (UID: \"d1527ada-72ef-4120-bc7c-22aaea22dc8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"
Sep 16 13:51:53.894436 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:53.894410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"
Sep 16 13:51:54.165595 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:54.165547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"]
Sep 16 13:51:54.168610 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:51:54.168579    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1527ada_72ef_4120_bc7c_22aaea22dc8f.slice/crio-4777c4131bca57ba50b6465cfe4df05cb63a9f11974725780c781c5e9b4021df WatchSource:0}: Error finding container 4777c4131bca57ba50b6465cfe4df05cb63a9f11974725780c781c5e9b4021df: Status 404 returned error can't find the container with id 4777c4131bca57ba50b6465cfe4df05cb63a9f11974725780c781c5e9b4021df
Sep 16 13:51:54.635992 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:54.635956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw" event={"ID":"d1527ada-72ef-4120-bc7c-22aaea22dc8f","Type":"ContainerStarted","Data":"e4f145f9e0f680b7648bffb90ba8a25ad6b558f331203a0051210ea5b064bb83"}
Sep 16 13:51:54.636359 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:54.636027    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw" event={"ID":"d1527ada-72ef-4120-bc7c-22aaea22dc8f","Type":"ContainerStarted","Data":"4777c4131bca57ba50b6465cfe4df05cb63a9f11974725780c781c5e9b4021df"}
Sep 16 13:51:54.652575 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:54.652511    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw" podStartSLOduration=1.6524905840000002 podStartE2EDuration="1.652490584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:51:54.649879073 +0000 UTC m=+1142.333537467" watchObservedRunningTime="2026-09-16 13:51:54.652490584 +0000 UTC m=+1142.336148977"
Sep 16 13:51:55.641834 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:55.641799    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1527ada-72ef-4120-bc7c-22aaea22dc8f" containerID="e4f145f9e0f680b7648bffb90ba8a25ad6b558f331203a0051210ea5b064bb83" exitCode=0
Sep 16 13:51:55.642248 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:55.641881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw" event={"ID":"d1527ada-72ef-4120-bc7c-22aaea22dc8f","Type":"ContainerDied","Data":"e4f145f9e0f680b7648bffb90ba8a25ad6b558f331203a0051210ea5b064bb83"}
Sep 16 13:51:56.777129 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:56.777104    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"
Sep 16 13:51:56.970778 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:56.970665    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1527ada-72ef-4120-bc7c-22aaea22dc8f-maas-api-service-ca\") pod \"d1527ada-72ef-4120-bc7c-22aaea22dc8f\" (UID: \"d1527ada-72ef-4120-bc7c-22aaea22dc8f\") "
Sep 16 13:51:56.971048 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:56.971024    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1527ada-72ef-4120-bc7c-22aaea22dc8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1527ada-72ef-4120-bc7c-22aaea22dc8f" (UID: "d1527ada-72ef-4120-bc7c-22aaea22dc8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:51:57.071858 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:57.071831    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1527ada-72ef-4120-bc7c-22aaea22dc8f-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:51:57.653164 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:57.653125    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"
Sep 16 13:51:57.653483 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:57.653163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw" event={"ID":"d1527ada-72ef-4120-bc7c-22aaea22dc8f","Type":"ContainerDied","Data":"4777c4131bca57ba50b6465cfe4df05cb63a9f11974725780c781c5e9b4021df"}
Sep 16 13:51:57.653483 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:57.653226    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4777c4131bca57ba50b6465cfe4df05cb63a9f11974725780c781c5e9b4021df"
Sep 16 13:51:58.607530 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:58.607501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:51:58.607872 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:51:58.607574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:52:04.421880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.421846    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"]
Sep 16 13:52:04.422441 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.422121    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk" podUID="bfc91291-a734-4134-97cb-c3ae166e4432" containerName="payload-processing" containerID="cri-o://a258cc5afe6cda4af485ea951060c346d4a00053d1088f3772993d3c6ac7928d" gracePeriod=30
Sep 16 13:52:04.429815 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.429763    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"]
Sep 16 13:52:04.430066 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.430021    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq" podUID="a62b43f4-4b67-4808-b4b1-12154db1d096" containerName="payload-pre-processing" containerID="cri-o://5deb824c119c79f0ae56030f578939305d6c23bd1e40e07adeec94eaa050e6f0" gracePeriod=30
Sep 16 13:52:04.584658 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.584572    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"]
Sep 16 13:52:04.587378 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.587349    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6c7792-zmjmw"]
Sep 16 13:52:04.691482 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.691452    2580 generic.go:356] "Generic (PLEG): container finished" podID="a62b43f4-4b67-4808-b4b1-12154db1d096" containerID="5deb824c119c79f0ae56030f578939305d6c23bd1e40e07adeec94eaa050e6f0" exitCode=0
Sep 16 13:52:04.691605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.691527    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq" event={"ID":"a62b43f4-4b67-4808-b4b1-12154db1d096","Type":"ContainerDied","Data":"5deb824c119c79f0ae56030f578939305d6c23bd1e40e07adeec94eaa050e6f0"}
Sep 16 13:52:04.693204 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.693183    2580 generic.go:356] "Generic (PLEG): container finished" podID="bfc91291-a734-4134-97cb-c3ae166e4432" containerID="a258cc5afe6cda4af485ea951060c346d4a00053d1088f3772993d3c6ac7928d" exitCode=0
Sep 16 13:52:04.693332 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.693242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk" event={"ID":"bfc91291-a734-4134-97cb-c3ae166e4432","Type":"ContainerDied","Data":"a258cc5afe6cda4af485ea951060c346d4a00053d1088f3772993d3c6ac7928d"}
Sep 16 13:52:04.693332 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.693296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk" event={"ID":"bfc91291-a734-4134-97cb-c3ae166e4432","Type":"ContainerDied","Data":"46e354710b1262e92bc40601927d74120565e0be2f78dd675ca948422fbe4830"}
Sep 16 13:52:04.693332 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.693320    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46e354710b1262e92bc40601927d74120565e0be2f78dd675ca948422fbe4830"
Sep 16 13:52:04.712581 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.712557    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:52:04.715569 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.715547    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:52:04.727828 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.727806    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prsm7\" (UniqueName: \"kubernetes.io/projected/a62b43f4-4b67-4808-b4b1-12154db1d096-kube-api-access-prsm7\") pod \"a62b43f4-4b67-4808-b4b1-12154db1d096\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") "
Sep 16 13:52:04.727989 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.727843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a62b43f4-4b67-4808-b4b1-12154db1d096-plugins-config-volume\") pod \"a62b43f4-4b67-4808-b4b1-12154db1d096\" (UID: \"a62b43f4-4b67-4808-b4b1-12154db1d096\") "
Sep 16 13:52:04.727989 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.727861    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwbjl\" (UniqueName: \"kubernetes.io/projected/bfc91291-a734-4134-97cb-c3ae166e4432-kube-api-access-qwbjl\") pod \"bfc91291-a734-4134-97cb-c3ae166e4432\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") "
Sep 16 13:52:04.727989 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.727928    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc91291-a734-4134-97cb-c3ae166e4432-plugins-config-volume\") pod \"bfc91291-a734-4134-97cb-c3ae166e4432\" (UID: \"bfc91291-a734-4134-97cb-c3ae166e4432\") "
Sep 16 13:52:04.728234 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.728211    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a62b43f4-4b67-4808-b4b1-12154db1d096-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a62b43f4-4b67-4808-b4b1-12154db1d096" (UID: "a62b43f4-4b67-4808-b4b1-12154db1d096"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:52:04.728377 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.728330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfc91291-a734-4134-97cb-c3ae166e4432-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bfc91291-a734-4134-97cb-c3ae166e4432" (UID: "bfc91291-a734-4134-97cb-c3ae166e4432"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:52:04.732959 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.731227    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfc91291-a734-4134-97cb-c3ae166e4432-kube-api-access-qwbjl" (OuterVolumeSpecName: "kube-api-access-qwbjl") pod "bfc91291-a734-4134-97cb-c3ae166e4432" (UID: "bfc91291-a734-4134-97cb-c3ae166e4432"). InnerVolumeSpecName "kube-api-access-qwbjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:52:04.732959 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.731357    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a62b43f4-4b67-4808-b4b1-12154db1d096-kube-api-access-prsm7" (OuterVolumeSpecName: "kube-api-access-prsm7") pod "a62b43f4-4b67-4808-b4b1-12154db1d096" (UID: "a62b43f4-4b67-4808-b4b1-12154db1d096"). InnerVolumeSpecName "kube-api-access-prsm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:52:04.802381 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.802352    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1527ada-72ef-4120-bc7c-22aaea22dc8f" path="/var/lib/kubelet/pods/d1527ada-72ef-4120-bc7c-22aaea22dc8f/volumes"
Sep 16 13:52:04.829182 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.829157    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a62b43f4-4b67-4808-b4b1-12154db1d096-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:52:04.829182 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.829182    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwbjl\" (UniqueName: \"kubernetes.io/projected/bfc91291-a734-4134-97cb-c3ae166e4432-kube-api-access-qwbjl\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:52:04.829319 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.829195    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfc91291-a734-4134-97cb-c3ae166e4432-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:52:04.829319 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:04.829206    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prsm7\" (UniqueName: \"kubernetes.io/projected/a62b43f4-4b67-4808-b4b1-12154db1d096-kube-api-access-prsm7\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:52:05.699111 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.699038    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"
Sep 16 13:52:05.699111 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.699066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq" event={"ID":"a62b43f4-4b67-4808-b4b1-12154db1d096","Type":"ContainerDied","Data":"464c1cc5cd6c02ba585d69e58b39c4826a1bf8b0c8d54773df9022e9cd214f2a"}
Sep 16 13:52:05.699606 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.699134    2580 scope.go:117] "RemoveContainer" containerID="5deb824c119c79f0ae56030f578939305d6c23bd1e40e07adeec94eaa050e6f0"
Sep 16 13:52:05.699606 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.699259    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"
Sep 16 13:52:05.729545 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.729522    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"]
Sep 16 13:52:05.736083 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.736062    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6c7792-d4cb9b4fd-7qdrq"]
Sep 16 13:52:05.767195 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.767171    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"]
Sep 16 13:52:05.772478 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:05.772458    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6c7792-5555bc7fff-wkqhk"]
Sep 16 13:52:06.802061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:06.802033    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a62b43f4-4b67-4808-b4b1-12154db1d096" path="/var/lib/kubelet/pods/a62b43f4-4b67-4808-b4b1-12154db1d096/volumes"
Sep 16 13:52:06.802427 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:06.802404    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfc91291-a734-4134-97cb-c3ae166e4432" path="/var/lib/kubelet/pods/bfc91291-a734-4134-97cb-c3ae166e4432/volumes"
Sep 16 13:52:12.473631 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.473595    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7692-ca110d"]
Sep 16 13:52:12.474021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.473958    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a62b43f4-4b67-4808-b4b1-12154db1d096" containerName="payload-pre-processing"
Sep 16 13:52:12.474021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.473972    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a62b43f4-4b67-4808-b4b1-12154db1d096" containerName="payload-pre-processing"
Sep 16 13:52:12.474021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.473980    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfc91291-a734-4134-97cb-c3ae166e4432" containerName="payload-processing"
Sep 16 13:52:12.474021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.473987    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfc91291-a734-4134-97cb-c3ae166e4432" containerName="payload-processing"
Sep 16 13:52:12.474021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.473999    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1527ada-72ef-4120-bc7c-22aaea22dc8f" containerName="revoke-keys"
Sep 16 13:52:12.474021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.474006    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1527ada-72ef-4120-bc7c-22aaea22dc8f" containerName="revoke-keys"
Sep 16 13:52:12.474211 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.474066    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1527ada-72ef-4120-bc7c-22aaea22dc8f" containerName="revoke-keys"
Sep 16 13:52:12.474211 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.474077    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a62b43f4-4b67-4808-b4b1-12154db1d096" containerName="payload-pre-processing"
Sep 16 13:52:12.474211 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.474085    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfc91291-a734-4134-97cb-c3ae166e4432" containerName="payload-processing"
Sep 16 13:52:12.478456 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.478438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:12.480135 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.480104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs79r\" (UniqueName: \"kubernetes.io/projected/0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b-kube-api-access-xs79r\") pod \"test-curl-7692-ca110d\" (UID: \"0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b\") " pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:12.480669 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.480644    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmn8d\""
Sep 16 13:52:12.482704 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.482665    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7692-ca110d"]
Sep 16 13:52:12.580531 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.580504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs79r\" (UniqueName: \"kubernetes.io/projected/0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b-kube-api-access-xs79r\") pod \"test-curl-7692-ca110d\" (UID: \"0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b\") " pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:12.587633 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.587607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs79r\" (UniqueName: \"kubernetes.io/projected/0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b-kube-api-access-xs79r\") pod \"test-curl-7692-ca110d\" (UID: \"0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b\") " pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:12.789242 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:12.789169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:13.054722 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:13.054508    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7692-ca110d"]
Sep 16 13:52:13.060052 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:13.060021    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ec31ded_55d2_4f5b_a8a7_bcb8ad17580b.slice/crio-bd01f0e4d7ab0619885fd516484009d123d63dff9a999cc0c6e0a7aa9f2e0a7d WatchSource:0}: Error finding container bd01f0e4d7ab0619885fd516484009d123d63dff9a999cc0c6e0a7aa9f2e0a7d: Status 404 returned error can't find the container with id bd01f0e4d7ab0619885fd516484009d123d63dff9a999cc0c6e0a7aa9f2e0a7d
Sep 16 13:52:13.740780 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:13.740745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7692-ca110d" event={"ID":"0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b","Type":"ContainerStarted","Data":"0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1"}
Sep 16 13:52:13.741164 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:13.740789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7692-ca110d" event={"ID":"0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b","Type":"ContainerStarted","Data":"bd01f0e4d7ab0619885fd516484009d123d63dff9a999cc0c6e0a7aa9f2e0a7d"}
Sep 16 13:52:13.754366 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:13.754320    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7692-ca110d" podStartSLOduration=1.754306376 podStartE2EDuration="1.754306376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:13.753312335 +0000 UTC m=+1161.436970728" watchObservedRunningTime="2026-09-16 13:52:13.754306376 +0000 UTC m=+1161.437964807"
Sep 16 13:52:14.389382 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.389347    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7692-ca110d"]
Sep 16 13:52:14.393422 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.393397    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7692-ca110d"]
Sep 16 13:52:14.508816 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.508731    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7692-7f556c"]
Sep 16 13:52:14.509184 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.509155    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" containerName="test-curl-7692-ca110d"
Sep 16 13:52:14.509184 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.509186    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" containerName="test-curl-7692-ca110d"
Sep 16 13:52:14.509339 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.509300    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" containerName="test-curl-7692-ca110d"
Sep 16 13:52:14.512586 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.512565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:14.516321 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.516298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7692-7f556c"]
Sep 16 13:52:14.594965 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.594933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtdbs\" (UniqueName: \"kubernetes.io/projected/8e698fa9-d356-4fc1-ad33-f058a6c0de4f-kube-api-access-wtdbs\") pod \"test-curl-7692-7f556c\" (UID: \"8e698fa9-d356-4fc1-ad33-f058a6c0de4f\") " pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:14.695713 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.695671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtdbs\" (UniqueName: \"kubernetes.io/projected/8e698fa9-d356-4fc1-ad33-f058a6c0de4f-kube-api-access-wtdbs\") pod \"test-curl-7692-7f556c\" (UID: \"8e698fa9-d356-4fc1-ad33-f058a6c0de4f\") " pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:14.703593 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.703562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtdbs\" (UniqueName: \"kubernetes.io/projected/8e698fa9-d356-4fc1-ad33-f058a6c0de4f-kube-api-access-wtdbs\") pod \"test-curl-7692-7f556c\" (UID: \"8e698fa9-d356-4fc1-ad33-f058a6c0de4f\") " pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:14.745757 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:14.745731    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7692-ca110d" container="test-curl-7692-ca110d"
Sep 16 13:52:14.823516 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:14.823444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:15.092378 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:15.092270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7692-7f556c"]
Sep 16 13:52:15.094856 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:15.094820    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e698fa9_d356_4fc1_ad33_f058a6c0de4f.slice/crio-3c66f060aa789163b9679d22950fd87e1669ee572580946b9c61b0bc3077c108 WatchSource:0}: Error finding container 3c66f060aa789163b9679d22950fd87e1669ee572580946b9c61b0bc3077c108: Status 404 returned error can't find the container with id 3c66f060aa789163b9679d22950fd87e1669ee572580946b9c61b0bc3077c108
Sep 16 13:52:15.751309 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:15.751270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7692-7f556c" event={"ID":"8e698fa9-d356-4fc1-ad33-f058a6c0de4f","Type":"ContainerStarted","Data":"c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4"}
Sep 16 13:52:15.751309 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:15.751317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7692-7f556c" event={"ID":"8e698fa9-d356-4fc1-ad33-f058a6c0de4f","Type":"ContainerStarted","Data":"3c66f060aa789163b9679d22950fd87e1669ee572580946b9c61b0bc3077c108"}
Sep 16 13:52:15.751775 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:15.751394    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7692-ca110d" podUID="0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" containerName="test-curl-7692-ca110d" containerID="cri-o://0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1" gracePeriod=2
Sep 16 13:52:15.766852 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:15.766800    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7692-7f556c" podStartSLOduration=1.766785208 podStartE2EDuration="1.766785208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:15.764528619 +0000 UTC m=+1163.448187024" watchObservedRunningTime="2026-09-16 13:52:15.766785208 +0000 UTC m=+1163.450443602"
Sep 16 13:52:16.352155 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.352118    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7692-7f556c"]
Sep 16 13:52:16.355768 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.355736    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7692-7f556c"]
Sep 16 13:52:16.479975 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.479871    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7692-679ca6"]
Sep 16 13:52:16.480332 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.480312    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e698fa9-d356-4fc1-ad33-f058a6c0de4f" containerName="test-curl-7692-7f556c"
Sep 16 13:52:16.480418 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.480335    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e698fa9-d356-4fc1-ad33-f058a6c0de4f" containerName="test-curl-7692-7f556c"
Sep 16 13:52:16.480478 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.480453    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e698fa9-d356-4fc1-ad33-f058a6c0de4f" containerName="test-curl-7692-7f556c"
Sep 16 13:52:16.483359 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.483337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-679ca6"
Sep 16 13:52:16.488162 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.488136    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7692-679ca6"]
Sep 16 13:52:16.510574 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.510550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z249\" (UniqueName: \"kubernetes.io/projected/314afe84-35d6-40ea-ab35-d286cf855336-kube-api-access-6z249\") pod \"test-curl-7692-679ca6\" (UID: \"314afe84-35d6-40ea-ab35-d286cf855336\") " pod="opendatahub/test-curl-7692-679ca6"
Sep 16 13:52:16.611470 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.611440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z249\" (UniqueName: \"kubernetes.io/projected/314afe84-35d6-40ea-ab35-d286cf855336-kube-api-access-6z249\") pod \"test-curl-7692-679ca6\" (UID: \"314afe84-35d6-40ea-ab35-d286cf855336\") " pod="opendatahub/test-curl-7692-679ca6"
Sep 16 13:52:16.623752 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.623724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z249\" (UniqueName: \"kubernetes.io/projected/314afe84-35d6-40ea-ab35-d286cf855336-kube-api-access-6z249\") pod \"test-curl-7692-679ca6\" (UID: \"314afe84-35d6-40ea-ab35-d286cf855336\") " pod="opendatahub/test-curl-7692-679ca6"
Sep 16 13:52:16.758272 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:16.758201    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7692-7f556c" container="test-curl-7692-7f556c"
Sep 16 13:52:16.793972 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:16.793947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-679ca6"
Sep 16 13:52:17.064357 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:17.064323    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7692-679ca6"]
Sep 16 13:52:17.066212 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:17.066186    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod314afe84_35d6_40ea_ab35_d286cf855336.slice/crio-1caf6b2ee453c516b149ffa44c288d850b2cab39c4515548f4a53d26778d2913 WatchSource:0}: Error finding container 1caf6b2ee453c516b149ffa44c288d850b2cab39c4515548f4a53d26778d2913: Status 404 returned error can't find the container with id 1caf6b2ee453c516b149ffa44c288d850b2cab39c4515548f4a53d26778d2913
Sep 16 13:52:17.766107 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:17.766079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7692-679ca6" event={"ID":"314afe84-35d6-40ea-ab35-d286cf855336","Type":"ContainerStarted","Data":"6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3"}
Sep 16 13:52:17.766427 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:17.766122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7692-679ca6" event={"ID":"314afe84-35d6-40ea-ab35-d286cf855336","Type":"ContainerStarted","Data":"1caf6b2ee453c516b149ffa44c288d850b2cab39c4515548f4a53d26778d2913"}
Sep 16 13:52:17.766427 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:17.766236    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7692-7f556c" podUID="8e698fa9-d356-4fc1-ad33-f058a6c0de4f" containerName="test-curl-7692-7f556c" containerID="cri-o://c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4" gracePeriod=2
Sep 16 13:52:17.781426 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:17.781383    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7692-679ca6" podStartSLOduration=1.7813671119999999 podStartE2EDuration="1.781367112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:17.779739638 +0000 UTC m=+1165.463398029" watchObservedRunningTime="2026-09-16 13:52:17.781367112 +0000 UTC m=+1165.465025506"
Sep 16 13:52:17.932161 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:17.932136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:18.025757 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.025688    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs79r\" (UniqueName: \"kubernetes.io/projected/0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b-kube-api-access-xs79r\") pod \"0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b\" (UID: \"0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b\") "
Sep 16 13:52:18.027778 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.027751    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b-kube-api-access-xs79r" (OuterVolumeSpecName: "kube-api-access-xs79r") pod "0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" (UID: "0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b"). InnerVolumeSpecName "kube-api-access-xs79r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:52:18.126467 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.126431    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs79r\" (UniqueName: \"kubernetes.io/projected/0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b-kube-api-access-xs79r\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:52:18.384118 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.384087    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7692-679ca6"]
Sep 16 13:52:18.389813 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.389787    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7692-679ca6"]
Sep 16 13:52:18.773400 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.773322    2580 generic.go:356] "Generic (PLEG): container finished" podID="0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" containerID="0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1" exitCode=137
Sep 16 13:52:18.773400 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.773377    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:18.773810 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.773429    2580 scope.go:117] "RemoveContainer" containerID="0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1"
Sep 16 13:52:18.773810 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:18.773685    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7692-679ca6" container="test-curl-7692-679ca6"
Sep 16 13:52:18.782460 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.782438    2580 scope.go:117] "RemoveContainer" containerID="0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1"
Sep 16 13:52:18.782814 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:18.782792    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1\": container with ID starting with 0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1 not found: ID does not exist" containerID="0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1"
Sep 16 13:52:18.782931 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.782823    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1"} err="failed to get container status \"0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1\": rpc error: code = NotFound desc = could not find container \"0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1\": container with ID starting with 0b2f02e04471b21ed825354ab53c655e9972d6c28dc5bf304e79f4fc3c494bd1 not found: ID does not exist"
Sep 16 13:52:18.783618 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:18.783590    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7692-ca110d\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" pod="opendatahub/test-curl-7692-ca110d"
Sep 16 13:52:18.808245 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:18.808216    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b" path="/var/lib/kubelet/pods/0ec31ded-55d2-4f5b-a8a7-bcb8ad17580b/volumes"
Sep 16 13:52:19.558105 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.558070    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"]
Sep 16 13:52:19.558408 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.558396    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314afe84-35d6-40ea-ab35-d286cf855336" containerName="test-curl-7692-679ca6"
Sep 16 13:52:19.558466 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.558409    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="314afe84-35d6-40ea-ab35-d286cf855336" containerName="test-curl-7692-679ca6"
Sep 16 13:52:19.558503 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.558482    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="314afe84-35d6-40ea-ab35-d286cf855336" containerName="test-curl-7692-679ca6"
Sep 16 13:52:19.562616 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.562598    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.564628 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.564604    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-2284bd\""
Sep 16 13:52:19.564760 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.564627    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-2284bd-dockercfg-rq2tx\""
Sep 16 13:52:19.571328 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.571306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"]
Sep 16 13:52:19.622256 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.622228    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"]
Sep 16 13:52:19.625423 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.625408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:19.634365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.634334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"]
Sep 16 13:52:19.638109 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.638079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.638202 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.638171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw4jt\" (UniqueName: \"kubernetes.io/projected/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-kube-api-access-gw4jt\") pod \"payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.638256 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.638222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns95t\" (UniqueName: \"kubernetes.io/projected/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-kube-api-access-ns95t\") pod \"payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:19.638310 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.638287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:19.739258 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.739221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.739412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.739281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw4jt\" (UniqueName: \"kubernetes.io/projected/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-kube-api-access-gw4jt\") pod \"payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.739412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.739315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns95t\" (UniqueName: \"kubernetes.io/projected/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-kube-api-access-ns95t\") pod \"payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:19.739412 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.739358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:19.740120 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.740098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:19.740207 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.740184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.746847 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.746821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw4jt\" (UniqueName: \"kubernetes.io/projected/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-kube-api-access-gw4jt\") pod \"payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.746980 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.746962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns95t\" (UniqueName: \"kubernetes.io/projected/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-kube-api-access-ns95t\") pod \"payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:19.779334 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.779306    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7692-679ca6" podUID="314afe84-35d6-40ea-ab35-d286cf855336" containerName="test-curl-7692-679ca6" containerID="cri-o://6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3" gracePeriod=2
Sep 16 13:52:19.873228 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.873165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:19.934717 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.934673    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:19.936285 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:19.936256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:20.042174 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.042143    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtdbs\" (UniqueName: \"kubernetes.io/projected/8e698fa9-d356-4fc1-ad33-f058a6c0de4f-kube-api-access-wtdbs\") pod \"8e698fa9-d356-4fc1-ad33-f058a6c0de4f\" (UID: \"8e698fa9-d356-4fc1-ad33-f058a6c0de4f\") "
Sep 16 13:52:20.044256 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.044226    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e698fa9-d356-4fc1-ad33-f058a6c0de4f-kube-api-access-wtdbs" (OuterVolumeSpecName: "kube-api-access-wtdbs") pod "8e698fa9-d356-4fc1-ad33-f058a6c0de4f" (UID: "8e698fa9-d356-4fc1-ad33-f058a6c0de4f"). InnerVolumeSpecName "kube-api-access-wtdbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:52:20.143193 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.143164    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtdbs\" (UniqueName: \"kubernetes.io/projected/8e698fa9-d356-4fc1-ad33-f058a6c0de4f-kube-api-access-wtdbs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:52:20.152294 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.152270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"]
Sep 16 13:52:20.154535 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:20.154511    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba6a7e9a_3d2b_435e_9dc9_e459ec176cb6.slice/crio-cfb163c70f1fc4625545e200787c3af55b682bdd9f8e3757931365c8f0f30c06 WatchSource:0}: Error finding container cfb163c70f1fc4625545e200787c3af55b682bdd9f8e3757931365c8f0f30c06: Status 404 returned error can't find the container with id cfb163c70f1fc4625545e200787c3af55b682bdd9f8e3757931365c8f0f30c06
Sep 16 13:52:20.212663 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.212638    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"]
Sep 16 13:52:20.213709 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:20.213677    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e16ec0_2f2b_4209_b9ac_f854d9c46e5c.slice/crio-1ab5ebd3a32fd0427280462a50752dbc331dcb35995bb37d12d6a1e9f50fdedd WatchSource:0}: Error finding container 1ab5ebd3a32fd0427280462a50752dbc331dcb35995bb37d12d6a1e9f50fdedd: Status 404 returned error can't find the container with id 1ab5ebd3a32fd0427280462a50752dbc331dcb35995bb37d12d6a1e9f50fdedd
Sep 16 13:52:20.786451 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.786413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n" event={"ID":"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c","Type":"ContainerStarted","Data":"b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5"}
Sep 16 13:52:20.786931 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.786554    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:20.786931 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.786594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n" event={"ID":"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c","Type":"ContainerStarted","Data":"1ab5ebd3a32fd0427280462a50752dbc331dcb35995bb37d12d6a1e9f50fdedd"}
Sep 16 13:52:20.787747 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.787719    2580 generic.go:356] "Generic (PLEG): container finished" podID="8e698fa9-d356-4fc1-ad33-f058a6c0de4f" containerID="c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4" exitCode=137
Sep 16 13:52:20.787852 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.787754    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:20.787852 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.787811    2580 scope.go:117] "RemoveContainer" containerID="c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4"
Sep 16 13:52:20.789604 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.789578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w" event={"ID":"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6","Type":"ContainerStarted","Data":"86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36"}
Sep 16 13:52:20.789725 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.789623    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w" event={"ID":"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6","Type":"ContainerStarted","Data":"cfb163c70f1fc4625545e200787c3af55b682bdd9f8e3757931365c8f0f30c06"}
Sep 16 13:52:20.789771 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.789727    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:20.797675 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.797657    2580 scope.go:117] "RemoveContainer" containerID="c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4"
Sep 16 13:52:20.797976 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:20.797955    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4\": container with ID starting with c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4 not found: ID does not exist" containerID="c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4"
Sep 16 13:52:20.798043 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.797987    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4"} err="failed to get container status \"c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4\": rpc error: code = NotFound desc = could not find container \"c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4\": container with ID starting with c0eeb1195fb564a8240a1f5e359d942d0d518d4e74d4203fc2061284f7c14dc4 not found: ID does not exist"
Sep 16 13:52:20.802089 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:20.802061    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7692-7f556c\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="8e698fa9-d356-4fc1-ad33-f058a6c0de4f" pod="opendatahub/test-curl-7692-7f556c"
Sep 16 13:52:20.802298 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.802251    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n" podStartSLOduration=1.8022342930000002 podStartE2EDuration="1.802234293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:20.800624665 +0000 UTC m=+1168.484283060" watchObservedRunningTime="2026-09-16 13:52:20.802234293 +0000 UTC m=+1168.485892688"
Sep 16 13:52:20.804954 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.804931    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e698fa9-d356-4fc1-ad33-f058a6c0de4f" path="/var/lib/kubelet/pods/8e698fa9-d356-4fc1-ad33-f058a6c0de4f/volumes"
Sep 16 13:52:20.818283 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:20.818248    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w" podStartSLOduration=1.8182367099999999 podStartE2EDuration="1.81823671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:20.816946172 +0000 UTC m=+1168.500604564" watchObservedRunningTime="2026-09-16 13:52:20.81823671 +0000 UTC m=+1168.501895103"
Sep 16 13:52:21.921872 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:21.921847    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-679ca6"
Sep 16 13:52:21.959702 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:21.959672    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z249\" (UniqueName: \"kubernetes.io/projected/314afe84-35d6-40ea-ab35-d286cf855336-kube-api-access-6z249\") pod \"314afe84-35d6-40ea-ab35-d286cf855336\" (UID: \"314afe84-35d6-40ea-ab35-d286cf855336\") "
Sep 16 13:52:21.961611 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:21.961586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/314afe84-35d6-40ea-ab35-d286cf855336-kube-api-access-6z249" (OuterVolumeSpecName: "kube-api-access-6z249") pod "314afe84-35d6-40ea-ab35-d286cf855336" (UID: "314afe84-35d6-40ea-ab35-d286cf855336"). InnerVolumeSpecName "kube-api-access-6z249". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:52:22.060935 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:22.060846    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z249\" (UniqueName: \"kubernetes.io/projected/314afe84-35d6-40ea-ab35-d286cf855336-kube-api-access-6z249\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:52:22.808567 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:22.808526    2580 generic.go:356] "Generic (PLEG): container finished" podID="314afe84-35d6-40ea-ab35-d286cf855336" containerID="6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3" exitCode=137
Sep 16 13:52:22.808758 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:22.808626    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7692-679ca6"
Sep 16 13:52:22.808758 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:22.808676    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314afe84-35d6-40ea-ab35-d286cf855336" path="/var/lib/kubelet/pods/314afe84-35d6-40ea-ab35-d286cf855336/volumes"
Sep 16 13:52:22.809210 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:22.809192    2580 scope.go:117] "RemoveContainer" containerID="6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3"
Sep 16 13:52:22.820064 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:22.819889    2580 scope.go:117] "RemoveContainer" containerID="6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3"
Sep 16 13:52:22.820245 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:52:22.820225    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3\": container with ID starting with 6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3 not found: ID does not exist" containerID="6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3"
Sep 16 13:52:22.820329 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:22.820251    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3"} err="failed to get container status \"6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3\": rpc error: code = NotFound desc = could not find container \"6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3\": container with ID starting with 6276c59c7c57c617ad9e52413feb75146011f2a72e20ccf50903e80e9ce9bdd3 not found: ID does not exist"
Sep 16 13:52:27.787368 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:27.787336    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"]
Sep 16 13:52:27.791831 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:27.791812    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:27.793946 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:27.793921    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3be042d3-dockercfg-6nvn4\""
Sep 16 13:52:27.794040 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:27.793949    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3be042d3\""
Sep 16 13:52:27.805135 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:27.805111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"]
Sep 16 13:52:27.906258 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:27.906225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj6bl\" (UniqueName: \"kubernetes.io/projected/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-kube-api-access-wj6bl\") pod \"payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") " pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:27.906394 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:27.906339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-plugins-config-volume\") pod \"payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") " pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:28.007775 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.007739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-plugins-config-volume\") pod \"payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") " pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:28.007942 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.007875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj6bl\" (UniqueName: \"kubernetes.io/projected/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-kube-api-access-wj6bl\") pod \"payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") " pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:28.008561 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.008542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-plugins-config-volume\") pod \"payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") " pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:28.014986 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.014963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj6bl\" (UniqueName: \"kubernetes.io/projected/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-kube-api-access-wj6bl\") pod \"payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") " pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:28.102416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.102341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:28.372647 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.372569    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"]
Sep 16 13:52:28.376071 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:28.376041    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4921e4df_e7e0_438a_9b5a_25efb6c31d2d.slice/crio-d30ff9c44930d856073d20e42d860318009f325253f28ddf25f314dc7bedcf76 WatchSource:0}: Error finding container d30ff9c44930d856073d20e42d860318009f325253f28ddf25f314dc7bedcf76: Status 404 returned error can't find the container with id d30ff9c44930d856073d20e42d860318009f325253f28ddf25f314dc7bedcf76
Sep 16 13:52:28.839312 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.839270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5" event={"ID":"4921e4df-e7e0-438a-9b5a-25efb6c31d2d","Type":"ContainerStarted","Data":"34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db"}
Sep 16 13:52:28.839676 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.839349    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5" event={"ID":"4921e4df-e7e0-438a-9b5a-25efb6c31d2d","Type":"ContainerStarted","Data":"d30ff9c44930d856073d20e42d860318009f325253f28ddf25f314dc7bedcf76"}
Sep 16 13:52:28.839676 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.839377    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:28.853724 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:28.853654    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5" podStartSLOduration=1.853641513 podStartE2EDuration="1.853641513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:28.853305875 +0000 UTC m=+1176.536964270" watchObservedRunningTime="2026-09-16 13:52:28.853641513 +0000 UTC m=+1176.537299907"
Sep 16 13:52:31.799211 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:31.799180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 13:52:31.799621 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:31.799365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 13:52:39.348460 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.348422    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"]
Sep 16 13:52:39.351464 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.351444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.353492 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.353472    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-ac8045\""
Sep 16 13:52:39.353605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.353476    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-ac8045-dockercfg-5dcxq\""
Sep 16 13:52:39.362396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.362374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"]
Sep 16 13:52:39.375764 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.375733    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"]
Sep 16 13:52:39.379430 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.379404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.388882 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.388852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"]
Sep 16 13:52:39.407818 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.407781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.408005 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.407835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc47q\" (UniqueName: \"kubernetes.io/projected/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-kube-api-access-fc47q\") pod \"payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.509386 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.509342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc47q\" (UniqueName: \"kubernetes.io/projected/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-kube-api-access-fc47q\") pod \"payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.509583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.509408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c42fd23f-7c49-496d-9cf8-37ead6d79045-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.509583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.509510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.509583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.509540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sgjc\" (UniqueName: \"kubernetes.io/projected/c42fd23f-7c49-496d-9cf8-37ead6d79045-kube-api-access-9sgjc\") pod \"payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.510217 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.510197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.516449 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.516419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc47q\" (UniqueName: \"kubernetes.io/projected/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-kube-api-access-fc47q\") pod \"payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.610799 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.610690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c42fd23f-7c49-496d-9cf8-37ead6d79045-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.610985 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.610873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sgjc\" (UniqueName: \"kubernetes.io/projected/c42fd23f-7c49-496d-9cf8-37ead6d79045-kube-api-access-9sgjc\") pod \"payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.611512 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.611491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c42fd23f-7c49-496d-9cf8-37ead6d79045-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.619199 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.619167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sgjc\" (UniqueName: \"kubernetes.io/projected/c42fd23f-7c49-496d-9cf8-37ead6d79045-kube-api-access-9sgjc\") pod \"payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.662526 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.662490    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:39.690093 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.690057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:39.846928 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.846874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:52:39.941657 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.941631    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"]
Sep 16 13:52:39.943995 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:39.943970    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee2f3a80_d462_4fa1_b3b8_5fb100a67b9d.slice/crio-285be5d054e34ae47d239c829255ceec021a0b3b8d863cf2e3e923d84a837621 WatchSource:0}: Error finding container 285be5d054e34ae47d239c829255ceec021a0b3b8d863cf2e3e923d84a837621: Status 404 returned error can't find the container with id 285be5d054e34ae47d239c829255ceec021a0b3b8d863cf2e3e923d84a837621
Sep 16 13:52:39.971180 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:39.971148    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"]
Sep 16 13:52:39.979932 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:52:39.979890    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42fd23f_7c49_496d_9cf8_37ead6d79045.slice/crio-b4c665698e9aa203a2e2777d26feb36c978ff2d6440b7199f5a07e94b4b06ce5 WatchSource:0}: Error finding container b4c665698e9aa203a2e2777d26feb36c978ff2d6440b7199f5a07e94b4b06ce5: Status 404 returned error can't find the container with id b4c665698e9aa203a2e2777d26feb36c978ff2d6440b7199f5a07e94b4b06ce5
Sep 16 13:52:40.904337 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.904296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46" event={"ID":"c42fd23f-7c49-496d-9cf8-37ead6d79045","Type":"ContainerStarted","Data":"760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc"}
Sep 16 13:52:40.904759 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.904354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46" event={"ID":"c42fd23f-7c49-496d-9cf8-37ead6d79045","Type":"ContainerStarted","Data":"b4c665698e9aa203a2e2777d26feb36c978ff2d6440b7199f5a07e94b4b06ce5"}
Sep 16 13:52:40.904759 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.904416    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:40.905950 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.905920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp" event={"ID":"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d","Type":"ContainerStarted","Data":"4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd"}
Sep 16 13:52:40.906042 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.905974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp" event={"ID":"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d","Type":"ContainerStarted","Data":"285be5d054e34ae47d239c829255ceec021a0b3b8d863cf2e3e923d84a837621"}
Sep 16 13:52:40.906042 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.906010    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:40.919708 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.919660    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46" podStartSLOduration=1.919644236 podStartE2EDuration="1.919644236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:40.918000696 +0000 UTC m=+1188.601659090" watchObservedRunningTime="2026-09-16 13:52:40.919644236 +0000 UTC m=+1188.603302630"
Sep 16 13:52:40.933295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:40.933249    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp" podStartSLOduration=1.9332316569999999 podStartE2EDuration="1.933231657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:52:40.932234945 +0000 UTC m=+1188.615893350" watchObservedRunningTime="2026-09-16 13:52:40.933231657 +0000 UTC m=+1188.616890051"
Sep 16 13:52:51.912737 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:51.912705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 13:52:51.913132 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:51.912860    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 13:52:52.749638 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:52.749609    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:52:52.754835 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:52.754799    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:52:52.862945 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:52:52.862917    2580 scope.go:117] "RemoveContainer" containerID="a258cc5afe6cda4af485ea951060c346d4a00053d1088f3772993d3c6ac7928d"
Sep 16 13:54:06.052308 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.052216    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"]
Sep 16 13:54:06.056487 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.056463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.059248 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.059220    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2284bd\"/\"default-dockercfg-pds95\""
Sep 16 13:54:06.059389 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.059262    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2284bd\"/\"openshift-service-ca.crt\""
Sep 16 13:54:06.059389 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.059293    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2284bd\"/\"auth-test-model-2284bd-kserve-self-signed-certs\""
Sep 16 13:54:06.059389 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.059263    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2284bd\"/\"kube-root-ca.crt\""
Sep 16 13:54:06.063981 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.063956    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"]
Sep 16 13:54:06.091859 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.091830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-home\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.092034 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.091868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tls-certs\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.092034 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.091891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-model-cache\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.092109 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.092051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tmp-dir\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.092109 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.092074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-dshm\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.092109 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.092092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcrwl\" (UniqueName: \"kubernetes.io/projected/a9f046f0-d92b-49c3-bfe0-7bb86208079b-kube-api-access-kcrwl\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193102 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-model-cache\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tmp-dir\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-dshm\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193349 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcrwl\" (UniqueName: \"kubernetes.io/projected/a9f046f0-d92b-49c3-bfe0-7bb86208079b-kube-api-access-kcrwl\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193349 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-home\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193426 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tls-certs\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193465 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193444    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-model-cache\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193587 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tmp-dir\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.193622 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.193607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-home\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.195515 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.195487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-dshm\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.195643 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.195628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tls-certs\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.202157 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.202129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcrwl\" (UniqueName: \"kubernetes.io/projected/a9f046f0-d92b-49c3-bfe0-7bb86208079b-kube-api-access-kcrwl\") pod \"auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") " pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.368184 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.368097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:06.638551 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:06.638523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"]
Sep 16 13:54:06.641167 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:54:06.641136    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9f046f0_d92b_49c3_bfe0_7bb86208079b.slice/crio-c967dbf04ca00a49514d028e49bf82e1798861040686d69a5317a47a00573c8b WatchSource:0}: Error finding container c967dbf04ca00a49514d028e49bf82e1798861040686d69a5317a47a00573c8b: Status 404 returned error can't find the container with id c967dbf04ca00a49514d028e49bf82e1798861040686d69a5317a47a00573c8b
Sep 16 13:54:07.334374 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:07.334333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5" event={"ID":"a9f046f0-d92b-49c3-bfe0-7bb86208079b","Type":"ContainerStarted","Data":"d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24"}
Sep 16 13:54:07.334783 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:07.334387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5" event={"ID":"a9f046f0-d92b-49c3-bfe0-7bb86208079b","Type":"ContainerStarted","Data":"c967dbf04ca00a49514d028e49bf82e1798861040686d69a5317a47a00573c8b"}
Sep 16 13:54:07.350758 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:07.350708    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5" podStartSLOduration=1.350684347 podStartE2EDuration="1.350684347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:54:07.349097461 +0000 UTC m=+1275.032755856" watchObservedRunningTime="2026-09-16 13:54:07.350684347 +0000 UTC m=+1275.034342783"
Sep 16 13:54:16.368999 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:16.368958    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:16.369507 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:16.369011    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:16.382350 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:16.382317    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:16.393922 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:16.393884    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:54:50.140522 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.140485    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"]
Sep 16 13:54:50.143661 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.143644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"
Sep 16 13:54:50.145599 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.145575    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:54:50.145681 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.145620    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:54:50.150977 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.150949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"]
Sep 16 13:54:50.190747 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.190720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ed87d1-4811-419e-9914-a437b657af8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz\" (UID: \"c7ed87d1-4811-419e-9914-a437b657af8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"
Sep 16 13:54:50.292087 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.292052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ed87d1-4811-419e-9914-a437b657af8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz\" (UID: \"c7ed87d1-4811-419e-9914-a437b657af8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"
Sep 16 13:54:50.292768 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.292749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ed87d1-4811-419e-9914-a437b657af8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz\" (UID: \"c7ed87d1-4811-419e-9914-a437b657af8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"
Sep 16 13:54:50.455399 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.455305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"
Sep 16 13:54:50.724442 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:50.724224    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"]
Sep 16 13:54:50.727212 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:54:50.727173    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ed87d1_4811_419e_9914_a437b657af8b.slice/crio-68322b4db53880b83cf436b7dc873ecbbe9771c342084fea220aae4526df0fe1 WatchSource:0}: Error finding container 68322b4db53880b83cf436b7dc873ecbbe9771c342084fea220aae4526df0fe1: Status 404 returned error can't find the container with id 68322b4db53880b83cf436b7dc873ecbbe9771c342084fea220aae4526df0fe1
Sep 16 13:54:51.570688 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:51.570650    2580 generic.go:356] "Generic (PLEG): container finished" podID="c7ed87d1-4811-419e-9914-a437b657af8b" containerID="9ae5528a66be80de00e80410c65687639ec757c8a576d4e571b38809fc1110da" exitCode=0
Sep 16 13:54:51.571142 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:51.570710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz" event={"ID":"c7ed87d1-4811-419e-9914-a437b657af8b","Type":"ContainerDied","Data":"9ae5528a66be80de00e80410c65687639ec757c8a576d4e571b38809fc1110da"}
Sep 16 13:54:51.571142 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:51.570753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz" event={"ID":"c7ed87d1-4811-419e-9914-a437b657af8b","Type":"ContainerStarted","Data":"68322b4db53880b83cf436b7dc873ecbbe9771c342084fea220aae4526df0fe1"}
Sep 16 13:54:52.708058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:52.708033    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"
Sep 16 13:54:52.818513 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:52.818483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ed87d1-4811-419e-9914-a437b657af8b-maas-api-service-ca\") pod \"c7ed87d1-4811-419e-9914-a437b657af8b\" (UID: \"c7ed87d1-4811-419e-9914-a437b657af8b\") "
Sep 16 13:54:52.818819 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:52.818799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7ed87d1-4811-419e-9914-a437b657af8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7ed87d1-4811-419e-9914-a437b657af8b" (UID: "c7ed87d1-4811-419e-9914-a437b657af8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:54:52.907261 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:52.907231    2580 scope.go:117] "RemoveContainer" containerID="ad2f4fb1d6b65c5a6c50bbc282a3ae0e4f7d664ace1a12549d17d460f5d3c38d"
Sep 16 13:54:52.919689 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:52.919660    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7ed87d1-4811-419e-9914-a437b657af8b-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:54:53.582089 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:53.582061    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"
Sep 16 13:54:53.582089 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:53.582072    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz" event={"ID":"c7ed87d1-4811-419e-9914-a437b657af8b","Type":"ContainerDied","Data":"68322b4db53880b83cf436b7dc873ecbbe9771c342084fea220aae4526df0fe1"}
Sep 16 13:54:53.582297 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:54:53.582115    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68322b4db53880b83cf436b7dc873ecbbe9771c342084fea220aae4526df0fe1"
Sep 16 13:55:05.701061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:05.701006    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"]
Sep 16 13:55:05.701873 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:05.701834    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5" podUID="4921e4df-e7e0-438a-9b5a-25efb6c31d2d" containerName="payload-processing" containerID="cri-o://34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db" gracePeriod=30
Sep 16 13:55:05.870748 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:05.870673    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"]
Sep 16 13:55:05.873712 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:05.873679    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3be042d3-9vqtz"]
Sep 16 13:55:05.964936 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:05.964853    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:55:06.051028 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.050996    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj6bl\" (UniqueName: \"kubernetes.io/projected/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-kube-api-access-wj6bl\") pod \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") "
Sep 16 13:55:06.051254 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.051078    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-plugins-config-volume\") pod \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\" (UID: \"4921e4df-e7e0-438a-9b5a-25efb6c31d2d\") "
Sep 16 13:55:06.051440 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.051417    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4921e4df-e7e0-438a-9b5a-25efb6c31d2d" (UID: "4921e4df-e7e0-438a-9b5a-25efb6c31d2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:55:06.053157 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.053119    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-kube-api-access-wj6bl" (OuterVolumeSpecName: "kube-api-access-wj6bl") pod "4921e4df-e7e0-438a-9b5a-25efb6c31d2d" (UID: "4921e4df-e7e0-438a-9b5a-25efb6c31d2d"). InnerVolumeSpecName "kube-api-access-wj6bl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:55:06.152293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.152255    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj6bl\" (UniqueName: \"kubernetes.io/projected/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-kube-api-access-wj6bl\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:55:06.152293 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.152288    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4921e4df-e7e0-438a-9b5a-25efb6c31d2d-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:55:06.654006 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.653969    2580 generic.go:356] "Generic (PLEG): container finished" podID="4921e4df-e7e0-438a-9b5a-25efb6c31d2d" containerID="34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db" exitCode=0
Sep 16 13:55:06.654189 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.654021    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"
Sep 16 13:55:06.654189 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.654059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5" event={"ID":"4921e4df-e7e0-438a-9b5a-25efb6c31d2d","Type":"ContainerDied","Data":"34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db"}
Sep 16 13:55:06.654189 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.654117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5" event={"ID":"4921e4df-e7e0-438a-9b5a-25efb6c31d2d","Type":"ContainerDied","Data":"d30ff9c44930d856073d20e42d860318009f325253f28ddf25f314dc7bedcf76"}
Sep 16 13:55:06.654189 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.654146    2580 scope.go:117] "RemoveContainer" containerID="34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db"
Sep 16 13:55:06.663915 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.663878    2580 scope.go:117] "RemoveContainer" containerID="34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db"
Sep 16 13:55:06.664227 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:55:06.664207    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db\": container with ID starting with 34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db not found: ID does not exist" containerID="34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db"
Sep 16 13:55:06.664275 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.664238    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db"} err="failed to get container status \"34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db\": rpc error: code = NotFound desc = could not find container \"34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db\": container with ID starting with 34053e17b58da7e679013532591181e37a4ca6298e86f7b8f3cf4f88ddfea3db not found: ID does not exist"
Sep 16 13:55:06.674945 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.674914    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"]
Sep 16 13:55:06.678603 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.678577    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3be042d3-7f5c57d965-gcfr5"]
Sep 16 13:55:06.803756 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.803725    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4921e4df-e7e0-438a-9b5a-25efb6c31d2d" path="/var/lib/kubelet/pods/4921e4df-e7e0-438a-9b5a-25efb6c31d2d/volumes"
Sep 16 13:55:06.804140 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:55:06.804119    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7ed87d1-4811-419e-9914-a437b657af8b" path="/var/lib/kubelet/pods/c7ed87d1-4811-419e-9914-a437b657af8b/volumes"
Sep 16 13:56:52.942027 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:52.941949    2580 scope.go:117] "RemoveContainer" containerID="30886d8df9ff0ac8fc110008e35a58da8bf71e2f425eb49afd8a6a0ebb325a3f"
Sep 16 13:56:58.592700 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.592665    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"]
Sep 16 13:56:58.593096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.593049    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4921e4df-e7e0-438a-9b5a-25efb6c31d2d" containerName="payload-processing"
Sep 16 13:56:58.593096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.593061    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4921e4df-e7e0-438a-9b5a-25efb6c31d2d" containerName="payload-processing"
Sep 16 13:56:58.593096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.593079    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ed87d1-4811-419e-9914-a437b657af8b" containerName="revoke-keys"
Sep 16 13:56:58.593096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.593086    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ed87d1-4811-419e-9914-a437b657af8b" containerName="revoke-keys"
Sep 16 13:56:58.593234 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.593140    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7ed87d1-4811-419e-9914-a437b657af8b" containerName="revoke-keys"
Sep 16 13:56:58.593234 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.593151    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4921e4df-e7e0-438a-9b5a-25efb6c31d2d" containerName="payload-processing"
Sep 16 13:56:58.597600 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.597578    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"
Sep 16 13:56:58.599609 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.599579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:56:58.599740 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.599706    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:56:58.603805 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.603784    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"]
Sep 16 13:56:58.746106 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.746067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0cd57-65ac-4a58-a92f-0e656ff88066-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78\" (UID: \"5dc0cd57-65ac-4a58-a92f-0e656ff88066\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"
Sep 16 13:56:58.846635 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.846549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0cd57-65ac-4a58-a92f-0e656ff88066-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78\" (UID: \"5dc0cd57-65ac-4a58-a92f-0e656ff88066\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"
Sep 16 13:56:58.847288 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.847261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0cd57-65ac-4a58-a92f-0e656ff88066-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78\" (UID: \"5dc0cd57-65ac-4a58-a92f-0e656ff88066\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"
Sep 16 13:56:58.909687 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:58.909650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"
Sep 16 13:56:59.187519 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:59.187469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"]
Sep 16 13:56:59.189795 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:56:59.189769    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc0cd57_65ac_4a58_a92f_0e656ff88066.slice/crio-32e629303a10832f2336030ec3376f4a650cb0f7f47635e32ee1e48d841249e4 WatchSource:0}: Error finding container 32e629303a10832f2336030ec3376f4a650cb0f7f47635e32ee1e48d841249e4: Status 404 returned error can't find the container with id 32e629303a10832f2336030ec3376f4a650cb0f7f47635e32ee1e48d841249e4
Sep 16 13:56:59.243729 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:56:59.243698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78" event={"ID":"5dc0cd57-65ac-4a58-a92f-0e656ff88066","Type":"ContainerStarted","Data":"32e629303a10832f2336030ec3376f4a650cb0f7f47635e32ee1e48d841249e4"}
Sep 16 13:57:00.250796 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:00.250706    2580 generic.go:356] "Generic (PLEG): container finished" podID="5dc0cd57-65ac-4a58-a92f-0e656ff88066" containerID="c1491827df124ef9e2eec84930172af5067e1e1aeb654276a778c848ed5d334f" exitCode=0
Sep 16 13:57:00.250796 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:00.250764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78" event={"ID":"5dc0cd57-65ac-4a58-a92f-0e656ff88066","Type":"ContainerDied","Data":"c1491827df124ef9e2eec84930172af5067e1e1aeb654276a778c848ed5d334f"}
Sep 16 13:57:01.381408 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:01.381384    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"
Sep 16 13:57:01.473492 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:01.473454    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0cd57-65ac-4a58-a92f-0e656ff88066-maas-api-service-ca\") pod \"5dc0cd57-65ac-4a58-a92f-0e656ff88066\" (UID: \"5dc0cd57-65ac-4a58-a92f-0e656ff88066\") "
Sep 16 13:57:01.473788 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:01.473762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dc0cd57-65ac-4a58-a92f-0e656ff88066-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5dc0cd57-65ac-4a58-a92f-0e656ff88066" (UID: "5dc0cd57-65ac-4a58-a92f-0e656ff88066"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:57:01.575184 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:01.575083    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0cd57-65ac-4a58-a92f-0e656ff88066-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:02.262746 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:02.262715    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"
Sep 16 13:57:02.262746 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:02.262739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78" event={"ID":"5dc0cd57-65ac-4a58-a92f-0e656ff88066","Type":"ContainerDied","Data":"32e629303a10832f2336030ec3376f4a650cb0f7f47635e32ee1e48d841249e4"}
Sep 16 13:57:02.263042 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:02.262785    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32e629303a10832f2336030ec3376f4a650cb0f7f47635e32ee1e48d841249e4"
Sep 16 13:57:03.468875 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:03.468836    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"]
Sep 16 13:57:03.485545 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:03.485513    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"]
Sep 16 13:57:13.859238 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.859197    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"]
Sep 16 13:57:13.859763 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.859741    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc0cd57-65ac-4a58-a92f-0e656ff88066" containerName="revoke-keys"
Sep 16 13:57:13.859852 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.859767    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc0cd57-65ac-4a58-a92f-0e656ff88066" containerName="revoke-keys"
Sep 16 13:57:13.859930 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.859884    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc0cd57-65ac-4a58-a92f-0e656ff88066" containerName="revoke-keys"
Sep 16 13:57:13.863084 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.863062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:13.864997 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.864968    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-44f000-dockercfg-wwnv9\""
Sep 16 13:57:13.865157 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.865137    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-44f000\""
Sep 16 13:57:13.871936 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.871887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h2wc\" (UniqueName: \"kubernetes.io/projected/6bbb063f-f505-4228-816a-2608deb1da94-kube-api-access-6h2wc\") pod \"payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:13.872031 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.872007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bbb063f-f505-4228-816a-2608deb1da94-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:13.876128 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.876104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"]
Sep 16 13:57:13.972949 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.972879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bbb063f-f505-4228-816a-2608deb1da94-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:13.973121 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.973005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h2wc\" (UniqueName: \"kubernetes.io/projected/6bbb063f-f505-4228-816a-2608deb1da94-kube-api-access-6h2wc\") pod \"payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:13.973582 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.973563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bbb063f-f505-4228-816a-2608deb1da94-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:13.981352 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:13.981326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h2wc\" (UniqueName: \"kubernetes.io/projected/6bbb063f-f505-4228-816a-2608deb1da94-kube-api-access-6h2wc\") pod \"payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:14.172801 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:14.172769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:14.445037 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:14.445011    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"]
Sep 16 13:57:14.447600 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:57:14.447573    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bbb063f_f505_4228_816a_2608deb1da94.slice/crio-9e05b1d3ab54cacf7944e8fabbc09b80cd82e32f5cd6a0b2477ed0e8e0c333ef WatchSource:0}: Error finding container 9e05b1d3ab54cacf7944e8fabbc09b80cd82e32f5cd6a0b2477ed0e8e0c333ef: Status 404 returned error can't find the container with id 9e05b1d3ab54cacf7944e8fabbc09b80cd82e32f5cd6a0b2477ed0e8e0c333ef
Sep 16 13:57:15.329841 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:15.329794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk" event={"ID":"6bbb063f-f505-4228-816a-2608deb1da94","Type":"ContainerStarted","Data":"a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582"}
Sep 16 13:57:15.330285 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:15.329859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk" event={"ID":"6bbb063f-f505-4228-816a-2608deb1da94","Type":"ContainerStarted","Data":"9e05b1d3ab54cacf7944e8fabbc09b80cd82e32f5cd6a0b2477ed0e8e0c333ef"}
Sep 16 13:57:15.330285 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:15.329983    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:15.347774 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:15.347725    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk" podStartSLOduration=2.3477093829999998 podStartE2EDuration="2.347709383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:57:15.345403277 +0000 UTC m=+1463.029061672" watchObservedRunningTime="2026-09-16 13:57:15.347709383 +0000 UTC m=+1463.031367778"
Sep 16 13:57:18.185649 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.185614    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"]
Sep 16 13:57:18.186113 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.185934    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj" podUID="d1377bcf-bfce-435a-8fe9-c2a5a394b23a" containerName="payload-processing" containerID="cri-o://69052424ef4ab70a923d21bc0d151fab5b29243a733e79bc532561a99295e20d" gracePeriod=30
Sep 16 13:57:18.195433 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.195403    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"]
Sep 16 13:57:18.195664 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.195637    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l" podUID="6b3ee91c-2ac1-4136-af65-d66ceeb8833a" containerName="payload-pre-processing" containerID="cri-o://f6a168a1c35e2f29097b0e367f468300c56b64324c1d2da7e61b68f546cb6d97" gracePeriod=30
Sep 16 13:57:18.348460 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.348425    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1377bcf-bfce-435a-8fe9-c2a5a394b23a" containerID="69052424ef4ab70a923d21bc0d151fab5b29243a733e79bc532561a99295e20d" exitCode=0
Sep 16 13:57:18.348640 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.348492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj" event={"ID":"d1377bcf-bfce-435a-8fe9-c2a5a394b23a","Type":"ContainerDied","Data":"69052424ef4ab70a923d21bc0d151fab5b29243a733e79bc532561a99295e20d"}
Sep 16 13:57:18.350713 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.350646    2580 generic.go:356] "Generic (PLEG): container finished" podID="6b3ee91c-2ac1-4136-af65-d66ceeb8833a" containerID="f6a168a1c35e2f29097b0e367f468300c56b64324c1d2da7e61b68f546cb6d97" exitCode=0
Sep 16 13:57:18.350713 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.350704    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l" event={"ID":"6b3ee91c-2ac1-4136-af65-d66ceeb8833a","Type":"ContainerDied","Data":"f6a168a1c35e2f29097b0e367f468300c56b64324c1d2da7e61b68f546cb6d97"}
Sep 16 13:57:18.354182 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.354156    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"]
Sep 16 13:57:18.356039 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.356013    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7d96c7-7wj78"]
Sep 16 13:57:18.479061 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.479033    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:57:18.482766 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.482742    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:57:18.488007 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.487983    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll" podUID="84128b40-1e4d-43c7-ad8f-fc3691f2fb63" containerName="main" containerID="cri-o://db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7" gracePeriod=45
Sep 16 13:57:18.504335 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.504303    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb" podUID="53d13dd8-ac40-42de-a40d-d48f581c47b2" containerName="main" containerID="cri-o://d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7" gracePeriod=45
Sep 16 13:57:18.508165 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.508131    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn5p8\" (UniqueName: \"kubernetes.io/projected/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-kube-api-access-rn5p8\") pod \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") "
Sep 16 13:57:18.508290 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.508217    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-plugins-config-volume\") pod \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") "
Sep 16 13:57:18.508290 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.508257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb9mm\" (UniqueName: \"kubernetes.io/projected/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-kube-api-access-mb9mm\") pod \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\" (UID: \"d1377bcf-bfce-435a-8fe9-c2a5a394b23a\") "
Sep 16 13:57:18.508413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.508325    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-plugins-config-volume\") pod \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\" (UID: \"6b3ee91c-2ac1-4136-af65-d66ceeb8833a\") "
Sep 16 13:57:18.508741 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.508697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1377bcf-bfce-435a-8fe9-c2a5a394b23a" (UID: "d1377bcf-bfce-435a-8fe9-c2a5a394b23a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:57:18.508863 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.508791    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b3ee91c-2ac1-4136-af65-d66ceeb8833a" (UID: "6b3ee91c-2ac1-4136-af65-d66ceeb8833a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:57:18.511057 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.511000    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-kube-api-access-rn5p8" (OuterVolumeSpecName: "kube-api-access-rn5p8") pod "6b3ee91c-2ac1-4136-af65-d66ceeb8833a" (UID: "6b3ee91c-2ac1-4136-af65-d66ceeb8833a"). InnerVolumeSpecName "kube-api-access-rn5p8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:57:18.511273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.511241    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-kube-api-access-mb9mm" (OuterVolumeSpecName: "kube-api-access-mb9mm") pod "d1377bcf-bfce-435a-8fe9-c2a5a394b23a" (UID: "d1377bcf-bfce-435a-8fe9-c2a5a394b23a"). InnerVolumeSpecName "kube-api-access-mb9mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:57:18.609529 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.609493    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn5p8\" (UniqueName: \"kubernetes.io/projected/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-kube-api-access-rn5p8\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.609529 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.609530    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.609712 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.609545    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb9mm\" (UniqueName: \"kubernetes.io/projected/d1377bcf-bfce-435a-8fe9-c2a5a394b23a-kube-api-access-mb9mm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.609712 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.609560    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b3ee91c-2ac1-4136-af65-d66ceeb8833a-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.764082 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.764059    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:57:18.768385 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.768364    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:57:18.807485 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.807449    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc0cd57-65ac-4a58-a92f-0e656ff88066" path="/var/lib/kubelet/pods/5dc0cd57-65ac-4a58-a92f-0e656ff88066/volumes"
Sep 16 13:57:18.810820 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.810791    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-home\") pod \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") "
Sep 16 13:57:18.810992 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.810836    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-model-cache\") pod \"53d13dd8-ac40-42de-a40d-d48f581c47b2\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") "
Sep 16 13:57:18.810992 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.810883    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-dshm\") pod \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") "
Sep 16 13:57:18.810992 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.810937    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-home\") pod \"53d13dd8-ac40-42de-a40d-d48f581c47b2\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") "
Sep 16 13:57:18.810992 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.810962    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d13dd8-ac40-42de-a40d-d48f581c47b2-tls-certs\") pod \"53d13dd8-ac40-42de-a40d-d48f581c47b2\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") "
Sep 16 13:57:18.811236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811012    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-tmp-dir\") pod \"53d13dd8-ac40-42de-a40d-d48f581c47b2\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") "
Sep 16 13:57:18.811236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811041    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p5vv\" (UniqueName: \"kubernetes.io/projected/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-kube-api-access-8p5vv\") pod \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") "
Sep 16 13:57:18.811236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-model-cache\") pod \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") "
Sep 16 13:57:18.811236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811065    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-home" (OuterVolumeSpecName: "home") pod "84128b40-1e4d-43c7-ad8f-fc3691f2fb63" (UID: "84128b40-1e4d-43c7-ad8f-fc3691f2fb63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.811236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811097    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tmp-dir\") pod \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") "
Sep 16 13:57:18.811236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811152    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-dshm\") pod \"53d13dd8-ac40-42de-a40d-d48f581c47b2\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") "
Sep 16 13:57:18.811236 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811188    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsnk4\" (UniqueName: \"kubernetes.io/projected/53d13dd8-ac40-42de-a40d-d48f581c47b2-kube-api-access-nsnk4\") pod \"53d13dd8-ac40-42de-a40d-d48f581c47b2\" (UID: \"53d13dd8-ac40-42de-a40d-d48f581c47b2\") "
Sep 16 13:57:18.811583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811245    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tls-certs\") pod \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\" (UID: \"84128b40-1e4d-43c7-ad8f-fc3691f2fb63\") "
Sep 16 13:57:18.811583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811302    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84128b40-1e4d-43c7-ad8f-fc3691f2fb63" (UID: "84128b40-1e4d-43c7-ad8f-fc3691f2fb63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.811583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811490    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "53d13dd8-ac40-42de-a40d-d48f581c47b2" (UID: "53d13dd8-ac40-42de-a40d-d48f581c47b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.811583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811509    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.811583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.811526    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.812756 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.812716    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "53d13dd8-ac40-42de-a40d-d48f581c47b2" (UID: "53d13dd8-ac40-42de-a40d-d48f581c47b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.813264 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.812990    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-model-cache" (OuterVolumeSpecName: "model-cache") pod "84128b40-1e4d-43c7-ad8f-fc3691f2fb63" (UID: "84128b40-1e4d-43c7-ad8f-fc3691f2fb63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.813475 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.813452    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-home" (OuterVolumeSpecName: "home") pod "53d13dd8-ac40-42de-a40d-d48f581c47b2" (UID: "53d13dd8-ac40-42de-a40d-d48f581c47b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.814243 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.814219    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84128b40-1e4d-43c7-ad8f-fc3691f2fb63" (UID: "84128b40-1e4d-43c7-ad8f-fc3691f2fb63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:57:18.814243 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.814224    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-kube-api-access-8p5vv" (OuterVolumeSpecName: "kube-api-access-8p5vv") pod "84128b40-1e4d-43c7-ad8f-fc3691f2fb63" (UID: "84128b40-1e4d-43c7-ad8f-fc3691f2fb63"). InnerVolumeSpecName "kube-api-access-8p5vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:57:18.815791 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.815744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-dshm" (OuterVolumeSpecName: "dshm") pod "84128b40-1e4d-43c7-ad8f-fc3691f2fb63" (UID: "84128b40-1e4d-43c7-ad8f-fc3691f2fb63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.815791 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.815749    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53d13dd8-ac40-42de-a40d-d48f581c47b2-kube-api-access-nsnk4" (OuterVolumeSpecName: "kube-api-access-nsnk4") pod "53d13dd8-ac40-42de-a40d-d48f581c47b2" (UID: "53d13dd8-ac40-42de-a40d-d48f581c47b2"). InnerVolumeSpecName "kube-api-access-nsnk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:57:18.816553 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.816521    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53d13dd8-ac40-42de-a40d-d48f581c47b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "53d13dd8-ac40-42de-a40d-d48f581c47b2" (UID: "53d13dd8-ac40-42de-a40d-d48f581c47b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:57:18.817104 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.817075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-dshm" (OuterVolumeSpecName: "dshm") pod "53d13dd8-ac40-42de-a40d-d48f581c47b2" (UID: "53d13dd8-ac40-42de-a40d-d48f581c47b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:57:18.912445 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912416    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912445 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912445    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912458    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912465    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912473    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d13dd8-ac40-42de-a40d-d48f581c47b2-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912480    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912488    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p5vv\" (UniqueName: \"kubernetes.io/projected/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-kube-api-access-8p5vv\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912498    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84128b40-1e4d-43c7-ad8f-fc3691f2fb63-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912507    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d13dd8-ac40-42de-a40d-d48f581c47b2-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:18.912636 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:18.912515    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsnk4\" (UniqueName: \"kubernetes.io/projected/53d13dd8-ac40-42de-a40d-d48f581c47b2-kube-api-access-nsnk4\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:57:19.357947 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.357886    2580 generic.go:356] "Generic (PLEG): container finished" podID="84128b40-1e4d-43c7-ad8f-fc3691f2fb63" containerID="db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7" exitCode=0
Sep 16 13:57:19.358394 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.357953    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"
Sep 16 13:57:19.358394 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.357971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll" event={"ID":"84128b40-1e4d-43c7-ad8f-fc3691f2fb63","Type":"ContainerDied","Data":"db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7"}
Sep 16 13:57:19.358394 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.358017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll" event={"ID":"84128b40-1e4d-43c7-ad8f-fc3691f2fb63","Type":"ContainerDied","Data":"111790159dca58e7704d229a3604b332eec1d38a433f7277bb287a3c0fc703d1"}
Sep 16 13:57:19.358394 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.358040    2580 scope.go:117] "RemoveContainer" containerID="db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7"
Sep 16 13:57:19.359536 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.359490    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"
Sep 16 13:57:19.359536 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.359492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l" event={"ID":"6b3ee91c-2ac1-4136-af65-d66ceeb8833a","Type":"ContainerDied","Data":"ecceb7bad655a132e2455157aaecc5f26000aa458a7d77e4a2df19ac313190bc"}
Sep 16 13:57:19.361096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.361067    2580 generic.go:356] "Generic (PLEG): container finished" podID="53d13dd8-ac40-42de-a40d-d48f581c47b2" containerID="d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7" exitCode=0
Sep 16 13:57:19.361238 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.361117    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"
Sep 16 13:57:19.361238 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.361147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb" event={"ID":"53d13dd8-ac40-42de-a40d-d48f581c47b2","Type":"ContainerDied","Data":"d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7"}
Sep 16 13:57:19.361238 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.361193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb" event={"ID":"53d13dd8-ac40-42de-a40d-d48f581c47b2","Type":"ContainerDied","Data":"5d195489d4d3822d0de82de0cdb4e40a3e1d922bc5c4246a82bcccd339eb1cc4"}
Sep 16 13:57:19.362930 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.362890    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"
Sep 16 13:57:19.362930 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.362924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj" event={"ID":"d1377bcf-bfce-435a-8fe9-c2a5a394b23a","Type":"ContainerDied","Data":"f27b794b33b5c0763c7fd503cf415d8f03bf75035ce3e7b1c4ed0db47ebedadf"}
Sep 16 13:57:19.368767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.368744    2580 scope.go:117] "RemoveContainer" containerID="db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7"
Sep 16 13:57:19.369096 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:57:19.369071    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7\": container with ID starting with db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7 not found: ID does not exist" containerID="db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7"
Sep 16 13:57:19.369186 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.369102    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7"} err="failed to get container status \"db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7\": rpc error: code = NotFound desc = could not find container \"db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7\": container with ID starting with db759b16e97bcd6acc286511c419270d9852a34ed5142d06143000a2fc89fdf7 not found: ID does not exist"
Sep 16 13:57:19.369186 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.369125    2580 scope.go:117] "RemoveContainer" containerID="f6a168a1c35e2f29097b0e367f468300c56b64324c1d2da7e61b68f546cb6d97"
Sep 16 13:57:19.377835 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.377817    2580 scope.go:117] "RemoveContainer" containerID="d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7"
Sep 16 13:57:19.379021 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.379001    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"]
Sep 16 13:57:19.382553 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.382530    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7d96c7-8bf8ccc9f-gj2sj"]
Sep 16 13:57:19.386380 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.386362    2580 scope.go:117] "RemoveContainer" containerID="d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7"
Sep 16 13:57:19.386690 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:57:19.386667    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7\": container with ID starting with d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7 not found: ID does not exist" containerID="d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7"
Sep 16 13:57:19.386735 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.386696    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7"} err="failed to get container status \"d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7\": rpc error: code = NotFound desc = could not find container \"d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7\": container with ID starting with d59d58c55f9e2864e0148a5d7d6085e7e797f1be3ed851d5df6a49ab667f81c7 not found: ID does not exist"
Sep 16 13:57:19.386735 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.386712    2580 scope.go:117] "RemoveContainer" containerID="69052424ef4ab70a923d21bc0d151fab5b29243a733e79bc532561a99295e20d"
Sep 16 13:57:19.395380 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.395350    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"]
Sep 16 13:57:19.400257 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.400235    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7d96c7-8585f964b-nm29l"]
Sep 16 13:57:19.410862 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.410838    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"]
Sep 16 13:57:19.413999 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.413977    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/distinct-w0-7d96c7-kserve-9b4b76f9c-8wwll"]
Sep 16 13:57:19.423440 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.423415    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"]
Sep 16 13:57:19.426826 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:19.426774    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-7d96c7/premium-w0-7d96c7-kserve-5875d69fc8-l8dbb"]
Sep 16 13:57:20.803088 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:20.803057    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d13dd8-ac40-42de-a40d-d48f581c47b2" path="/var/lib/kubelet/pods/53d13dd8-ac40-42de-a40d-d48f581c47b2/volumes"
Sep 16 13:57:20.803501 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:20.803419    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b3ee91c-2ac1-4136-af65-d66ceeb8833a" path="/var/lib/kubelet/pods/6b3ee91c-2ac1-4136-af65-d66ceeb8833a/volumes"
Sep 16 13:57:20.803761 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:20.803749    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84128b40-1e4d-43c7-ad8f-fc3691f2fb63" path="/var/lib/kubelet/pods/84128b40-1e4d-43c7-ad8f-fc3691f2fb63/volumes"
Sep 16 13:57:20.804089 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:20.804076    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1377bcf-bfce-435a-8fe9-c2a5a394b23a" path="/var/lib/kubelet/pods/d1377bcf-bfce-435a-8fe9-c2a5a394b23a/volumes"
Sep 16 13:57:26.337745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:26.337712    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:57:48.380713 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.380674    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"]
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381120    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84128b40-1e4d-43c7-ad8f-fc3691f2fb63" containerName="main"
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381134    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="84128b40-1e4d-43c7-ad8f-fc3691f2fb63" containerName="main"
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381144    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1377bcf-bfce-435a-8fe9-c2a5a394b23a" containerName="payload-processing"
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381150    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1377bcf-bfce-435a-8fe9-c2a5a394b23a" containerName="payload-processing"
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381157    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d13dd8-ac40-42de-a40d-d48f581c47b2" containerName="main"
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381162    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d13dd8-ac40-42de-a40d-d48f581c47b2" containerName="main"
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381174    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b3ee91c-2ac1-4136-af65-d66ceeb8833a" containerName="payload-pre-processing"
Sep 16 13:57:48.381226 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381180    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b3ee91c-2ac1-4136-af65-d66ceeb8833a" containerName="payload-pre-processing"
Sep 16 13:57:48.381511 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381239    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d13dd8-ac40-42de-a40d-d48f581c47b2" containerName="main"
Sep 16 13:57:48.381511 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381247    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="84128b40-1e4d-43c7-ad8f-fc3691f2fb63" containerName="main"
Sep 16 13:57:48.381511 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381258    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b3ee91c-2ac1-4136-af65-d66ceeb8833a" containerName="payload-pre-processing"
Sep 16 13:57:48.381511 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.381265    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1377bcf-bfce-435a-8fe9-c2a5a394b23a" containerName="payload-processing"
Sep 16 13:57:48.384382 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.384365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.386664 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.386636    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d6baf3-dockercfg-twxr5\""
Sep 16 13:57:48.386812 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.386637    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d6baf3\""
Sep 16 13:57:48.392295 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.392264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"]
Sep 16 13:57:48.492132 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.492095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr6ps\" (UniqueName: \"kubernetes.io/projected/351285ca-be57-4fa2-91ad-a24605ff3170-kube-api-access-fr6ps\") pod \"payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") " pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.492300 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.492262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351285ca-be57-4fa2-91ad-a24605ff3170-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") " pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.593268 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.593226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351285ca-be57-4fa2-91ad-a24605ff3170-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") " pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.593457 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.593277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr6ps\" (UniqueName: \"kubernetes.io/projected/351285ca-be57-4fa2-91ad-a24605ff3170-kube-api-access-fr6ps\") pod \"payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") " pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.594160 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.594136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351285ca-be57-4fa2-91ad-a24605ff3170-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") " pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.601353 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.601325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr6ps\" (UniqueName: \"kubernetes.io/projected/351285ca-be57-4fa2-91ad-a24605ff3170-kube-api-access-fr6ps\") pod \"payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") " pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.696349 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.696256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:48.966260 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:48.966233    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"]
Sep 16 13:57:48.969162 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:57:48.969135    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351285ca_be57_4fa2_91ad_a24605ff3170.slice/crio-89133186f6e1786d6bda026608e0623a4d49ffd4801af008034d736f1ded7277 WatchSource:0}: Error finding container 89133186f6e1786d6bda026608e0623a4d49ffd4801af008034d736f1ded7277: Status 404 returned error can't find the container with id 89133186f6e1786d6bda026608e0623a4d49ffd4801af008034d736f1ded7277
Sep 16 13:57:49.525952 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:49.525893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj" event={"ID":"351285ca-be57-4fa2-91ad-a24605ff3170","Type":"ContainerStarted","Data":"bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051"}
Sep 16 13:57:49.526424 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:49.525974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj" event={"ID":"351285ca-be57-4fa2-91ad-a24605ff3170","Type":"ContainerStarted","Data":"89133186f6e1786d6bda026608e0623a4d49ffd4801af008034d736f1ded7277"}
Sep 16 13:57:49.526424 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:49.526037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:57:49.542231 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:49.542181    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj" podStartSLOduration=1.542169346 podStartE2EDuration="1.542169346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:57:49.540749746 +0000 UTC m=+1497.224408153" watchObservedRunningTime="2026-09-16 13:57:49.542169346 +0000 UTC m=+1497.225827780"
Sep 16 13:57:52.780610 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:52.780579    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:57:52.783770 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:52.783745    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 13:57:52.961030 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:52.960995    2580 scope.go:117] "RemoveContainer" containerID="b889c743574048c13d4e80534adbdd658fb237dd8f312afeb08a1f28c806e957"
Sep 16 13:57:52.969583 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:57:52.969560    2580 scope.go:117] "RemoveContainer" containerID="7cd2102babd88016d350150866bb64c9bb02cd8fa387e8ecf4e75fd9a2b6de6c"
Sep 16 13:58:00.534214 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:00.534179    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:58:15.935155 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:15.935073    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"]
Sep 16 13:58:15.938536 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:15.938515    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"
Sep 16 13:58:15.940577 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:15.940555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:58:15.940682 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:15.940608    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:58:15.945616 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:15.945590    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"]
Sep 16 13:58:16.037439 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:16.037402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19acea3-4bd9-4541-b026-48573d8be883-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-44f000-pwzgj\" (UID: \"c19acea3-4bd9-4541-b026-48573d8be883\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"
Sep 16 13:58:16.138772 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:16.138735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19acea3-4bd9-4541-b026-48573d8be883-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-44f000-pwzgj\" (UID: \"c19acea3-4bd9-4541-b026-48573d8be883\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"
Sep 16 13:58:16.139390 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:16.139371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19acea3-4bd9-4541-b026-48573d8be883-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-44f000-pwzgj\" (UID: \"c19acea3-4bd9-4541-b026-48573d8be883\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"
Sep 16 13:58:16.251744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:16.251637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"
Sep 16 13:58:16.524098 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:16.523910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"]
Sep 16 13:58:16.526933 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:58:16.526885    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19acea3_4bd9_4541_b026_48573d8be883.slice/crio-193950c5b98c6391a9896ae4b05113124b6f9c915b07e1af5b34d1f405803e13 WatchSource:0}: Error finding container 193950c5b98c6391a9896ae4b05113124b6f9c915b07e1af5b34d1f405803e13: Status 404 returned error can't find the container with id 193950c5b98c6391a9896ae4b05113124b6f9c915b07e1af5b34d1f405803e13
Sep 16 13:58:16.666033 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:16.665995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj" event={"ID":"c19acea3-4bd9-4541-b026-48573d8be883","Type":"ContainerStarted","Data":"193950c5b98c6391a9896ae4b05113124b6f9c915b07e1af5b34d1f405803e13"}
Sep 16 13:58:17.672998 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:17.672961    2580 generic.go:356] "Generic (PLEG): container finished" podID="c19acea3-4bd9-4541-b026-48573d8be883" containerID="1972b6984d89864c449811fa892df1049d18068d3e011f48f62b711637cc65d3" exitCode=0
Sep 16 13:58:17.672998 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:17.673001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj" event={"ID":"c19acea3-4bd9-4541-b026-48573d8be883","Type":"ContainerDied","Data":"1972b6984d89864c449811fa892df1049d18068d3e011f48f62b711637cc65d3"}
Sep 16 13:58:18.815697 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:18.815668    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"
Sep 16 13:58:18.865874 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:18.865846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19acea3-4bd9-4541-b026-48573d8be883-maas-api-service-ca\") pod \"c19acea3-4bd9-4541-b026-48573d8be883\" (UID: \"c19acea3-4bd9-4541-b026-48573d8be883\") "
Sep 16 13:58:18.866259 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:18.866234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c19acea3-4bd9-4541-b026-48573d8be883-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c19acea3-4bd9-4541-b026-48573d8be883" (UID: "c19acea3-4bd9-4541-b026-48573d8be883"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:58:18.966863 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:18.966794    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19acea3-4bd9-4541-b026-48573d8be883-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:58:19.683322 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:19.683296    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"
Sep 16 13:58:19.683584 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:19.683290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj" event={"ID":"c19acea3-4bd9-4541-b026-48573d8be883","Type":"ContainerDied","Data":"193950c5b98c6391a9896ae4b05113124b6f9c915b07e1af5b34d1f405803e13"}
Sep 16 13:58:19.683584 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:19.683434    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="193950c5b98c6391a9896ae4b05113124b6f9c915b07e1af5b34d1f405803e13"
Sep 16 13:58:31.879620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:31.879583    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"]
Sep 16 13:58:31.882408 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:31.879826    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk" podUID="6bbb063f-f505-4228-816a-2608deb1da94" containerName="payload-pre-processing" containerID="cri-o://a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582" gracePeriod=30
Sep 16 13:58:32.040830 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.040762    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"]
Sep 16 13:58:32.043299 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.043274    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f000-pwzgj"]
Sep 16 13:58:32.137077 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.137010    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:58:32.283282 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.283246    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bbb063f-f505-4228-816a-2608deb1da94-plugins-config-volume\") pod \"6bbb063f-f505-4228-816a-2608deb1da94\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") "
Sep 16 13:58:32.283464 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.283318    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h2wc\" (UniqueName: \"kubernetes.io/projected/6bbb063f-f505-4228-816a-2608deb1da94-kube-api-access-6h2wc\") pod \"6bbb063f-f505-4228-816a-2608deb1da94\" (UID: \"6bbb063f-f505-4228-816a-2608deb1da94\") "
Sep 16 13:58:32.283605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.283577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bbb063f-f505-4228-816a-2608deb1da94-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bbb063f-f505-4228-816a-2608deb1da94" (UID: "6bbb063f-f505-4228-816a-2608deb1da94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:58:32.285429 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.285401    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bbb063f-f505-4228-816a-2608deb1da94-kube-api-access-6h2wc" (OuterVolumeSpecName: "kube-api-access-6h2wc") pod "6bbb063f-f505-4228-816a-2608deb1da94" (UID: "6bbb063f-f505-4228-816a-2608deb1da94"). InnerVolumeSpecName "kube-api-access-6h2wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:58:32.384605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.384570    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bbb063f-f505-4228-816a-2608deb1da94-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:58:32.384605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.384598    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h2wc\" (UniqueName: \"kubernetes.io/projected/6bbb063f-f505-4228-816a-2608deb1da94-kube-api-access-6h2wc\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:58:32.746222 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.746190    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bbb063f-f505-4228-816a-2608deb1da94" containerID="a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582" exitCode=0
Sep 16 13:58:32.746413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.746240    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"
Sep 16 13:58:32.746413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.746247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk" event={"ID":"6bbb063f-f505-4228-816a-2608deb1da94","Type":"ContainerDied","Data":"a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582"}
Sep 16 13:58:32.746413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.746351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk" event={"ID":"6bbb063f-f505-4228-816a-2608deb1da94","Type":"ContainerDied","Data":"9e05b1d3ab54cacf7944e8fabbc09b80cd82e32f5cd6a0b2477ed0e8e0c333ef"}
Sep 16 13:58:32.746413 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.746373    2580 scope.go:117] "RemoveContainer" containerID="a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582"
Sep 16 13:58:32.756425 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.756395    2580 scope.go:117] "RemoveContainer" containerID="a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582"
Sep 16 13:58:32.756694 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:58:32.756673    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582\": container with ID starting with a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582 not found: ID does not exist" containerID="a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582"
Sep 16 13:58:32.756749 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.756702    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582"} err="failed to get container status \"a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582\": rpc error: code = NotFound desc = could not find container \"a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582\": container with ID starting with a2b88e6281a6d33224b389630f8ed2df68128f919971a0dda7770df917ce4582 not found: ID does not exist"
Sep 16 13:58:32.769313 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.769290    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"]
Sep 16 13:58:32.774963 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.774943    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-44f000-b7677b874-b2llk"]
Sep 16 13:58:32.803987 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.803960    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bbb063f-f505-4228-816a-2608deb1da94" path="/var/lib/kubelet/pods/6bbb063f-f505-4228-816a-2608deb1da94/volumes"
Sep 16 13:58:32.804365 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:32.804348    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c19acea3-4bd9-4541-b026-48573d8be883" path="/var/lib/kubelet/pods/c19acea3-4bd9-4541-b026-48573d8be883/volumes"
Sep 16 13:58:38.948839 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.948803    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"]
Sep 16 13:58:38.949223 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.949191    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19acea3-4bd9-4541-b026-48573d8be883" containerName="revoke-keys"
Sep 16 13:58:38.949223 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.949204    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19acea3-4bd9-4541-b026-48573d8be883" containerName="revoke-keys"
Sep 16 13:58:38.949223 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.949222    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bbb063f-f505-4228-816a-2608deb1da94" containerName="payload-pre-processing"
Sep 16 13:58:38.949323 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.949227    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bbb063f-f505-4228-816a-2608deb1da94" containerName="payload-pre-processing"
Sep 16 13:58:38.949323 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.949284    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19acea3-4bd9-4541-b026-48573d8be883" containerName="revoke-keys"
Sep 16 13:58:38.949323 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.949295    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bbb063f-f505-4228-816a-2608deb1da94" containerName="payload-pre-processing"
Sep 16 13:58:38.953744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.953727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"
Sep 16 13:58:38.955810 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.955784    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:58:38.955945 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.955792    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:58:38.959685 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:38.959664    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"]
Sep 16 13:58:39.040516 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.040484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4b0547-2d4e-4c79-8b71-989ef2040dad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d6baf3-v245r\" (UID: \"7a4b0547-2d4e-4c79-8b71-989ef2040dad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"
Sep 16 13:58:39.141013 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.140981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4b0547-2d4e-4c79-8b71-989ef2040dad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d6baf3-v245r\" (UID: \"7a4b0547-2d4e-4c79-8b71-989ef2040dad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"
Sep 16 13:58:39.141605 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.141584    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4b0547-2d4e-4c79-8b71-989ef2040dad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d6baf3-v245r\" (UID: \"7a4b0547-2d4e-4c79-8b71-989ef2040dad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"
Sep 16 13:58:39.266058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.265978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"
Sep 16 13:58:39.543470 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.543439    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"]
Sep 16 13:58:39.545805 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:58:39.545775    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4b0547_2d4e_4c79_8b71_989ef2040dad.slice/crio-403db3270c71cdad78a0225fb189025784240fabbd955a5f2d98a015ad7aedb6 WatchSource:0}: Error finding container 403db3270c71cdad78a0225fb189025784240fabbd955a5f2d98a015ad7aedb6: Status 404 returned error can't find the container with id 403db3270c71cdad78a0225fb189025784240fabbd955a5f2d98a015ad7aedb6
Sep 16 13:58:39.783144 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.783112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r" event={"ID":"7a4b0547-2d4e-4c79-8b71-989ef2040dad","Type":"ContainerStarted","Data":"527b396d44302c127285777dea10578de79138423583cd1dbcbe5f39a39bdb3e"}
Sep 16 13:58:39.783314 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.783162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r" event={"ID":"7a4b0547-2d4e-4c79-8b71-989ef2040dad","Type":"ContainerStarted","Data":"403db3270c71cdad78a0225fb189025784240fabbd955a5f2d98a015ad7aedb6"}
Sep 16 13:58:39.796955 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:39.796842    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r" podStartSLOduration=1.796827288 podStartE2EDuration="1.796827288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:58:39.795972815 +0000 UTC m=+1547.479631209" watchObservedRunningTime="2026-09-16 13:58:39.796827288 +0000 UTC m=+1547.480485722"
Sep 16 13:58:40.789779 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:40.789743    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a4b0547-2d4e-4c79-8b71-989ef2040dad" containerID="527b396d44302c127285777dea10578de79138423583cd1dbcbe5f39a39bdb3e" exitCode=0
Sep 16 13:58:40.790168 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:40.789822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r" event={"ID":"7a4b0547-2d4e-4c79-8b71-989ef2040dad","Type":"ContainerDied","Data":"527b396d44302c127285777dea10578de79138423583cd1dbcbe5f39a39bdb3e"}
Sep 16 13:58:41.929240 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:41.929214    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"
Sep 16 13:58:42.068645 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:42.068552    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4b0547-2d4e-4c79-8b71-989ef2040dad-maas-api-service-ca\") pod \"7a4b0547-2d4e-4c79-8b71-989ef2040dad\" (UID: \"7a4b0547-2d4e-4c79-8b71-989ef2040dad\") "
Sep 16 13:58:42.068990 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:42.068960    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a4b0547-2d4e-4c79-8b71-989ef2040dad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a4b0547-2d4e-4c79-8b71-989ef2040dad" (UID: "7a4b0547-2d4e-4c79-8b71-989ef2040dad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:58:42.170129 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:42.170097    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4b0547-2d4e-4c79-8b71-989ef2040dad-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:58:42.801599 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:42.801575    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"
Sep 16 13:58:42.803221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:42.803192    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r" event={"ID":"7a4b0547-2d4e-4c79-8b71-989ef2040dad","Type":"ContainerDied","Data":"403db3270c71cdad78a0225fb189025784240fabbd955a5f2d98a015ad7aedb6"}
Sep 16 13:58:42.803351 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:42.803242    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="403db3270c71cdad78a0225fb189025784240fabbd955a5f2d98a015ad7aedb6"
Sep 16 13:58:53.008464 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:53.008436    2580 scope.go:117] "RemoveContainer" containerID="e4f145f9e0f680b7648bffb90ba8a25ad6b558f331203a0051210ea5b064bb83"
Sep 16 13:58:54.472079 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.472045    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"]
Sep 16 13:58:54.472523 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.472245    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj" podUID="351285ca-be57-4fa2-91ad-a24605ff3170" containerName="payload-processing" containerID="cri-o://bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051" gracePeriod=30
Sep 16 13:58:54.659016 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.658957    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"]
Sep 16 13:58:54.661549 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.661518    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d6baf3-v245r"]
Sep 16 13:58:54.721272 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.721247    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:58:54.765397 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.765324    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr6ps\" (UniqueName: \"kubernetes.io/projected/351285ca-be57-4fa2-91ad-a24605ff3170-kube-api-access-fr6ps\") pod \"351285ca-be57-4fa2-91ad-a24605ff3170\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") "
Sep 16 13:58:54.765527 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.765396    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351285ca-be57-4fa2-91ad-a24605ff3170-plugins-config-volume\") pod \"351285ca-be57-4fa2-91ad-a24605ff3170\" (UID: \"351285ca-be57-4fa2-91ad-a24605ff3170\") "
Sep 16 13:58:54.765787 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.765758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351285ca-be57-4fa2-91ad-a24605ff3170-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "351285ca-be57-4fa2-91ad-a24605ff3170" (UID: "351285ca-be57-4fa2-91ad-a24605ff3170"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:58:54.767406 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.767381    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351285ca-be57-4fa2-91ad-a24605ff3170-kube-api-access-fr6ps" (OuterVolumeSpecName: "kube-api-access-fr6ps") pod "351285ca-be57-4fa2-91ad-a24605ff3170" (UID: "351285ca-be57-4fa2-91ad-a24605ff3170"). InnerVolumeSpecName "kube-api-access-fr6ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:58:54.808040 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.808009    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a4b0547-2d4e-4c79-8b71-989ef2040dad" path="/var/lib/kubelet/pods/7a4b0547-2d4e-4c79-8b71-989ef2040dad/volumes"
Sep 16 13:58:54.857874 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.857843    2580 generic.go:356] "Generic (PLEG): container finished" podID="351285ca-be57-4fa2-91ad-a24605ff3170" containerID="bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051" exitCode=0
Sep 16 13:58:54.858024 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.857889    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"
Sep 16 13:58:54.858024 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.857929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj" event={"ID":"351285ca-be57-4fa2-91ad-a24605ff3170","Type":"ContainerDied","Data":"bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051"}
Sep 16 13:58:54.858024 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.857977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj" event={"ID":"351285ca-be57-4fa2-91ad-a24605ff3170","Type":"ContainerDied","Data":"89133186f6e1786d6bda026608e0623a4d49ffd4801af008034d736f1ded7277"}
Sep 16 13:58:54.858024 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.858005    2580 scope.go:117] "RemoveContainer" containerID="bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051"
Sep 16 13:58:54.866330 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.866312    2580 scope.go:117] "RemoveContainer" containerID="bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051"
Sep 16 13:58:54.866615 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:58:54.866593    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051\": container with ID starting with bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051 not found: ID does not exist" containerID="bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051"
Sep 16 13:58:54.866675 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.866630    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051"} err="failed to get container status \"bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051\": rpc error: code = NotFound desc = could not find container \"bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051\": container with ID starting with bd9ac5a0c22e44b8abbbb51bcca3761d3e63de7319dd78b6d57dd73cfe6cc051 not found: ID does not exist"
Sep 16 13:58:54.866675 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.866603    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351285ca-be57-4fa2-91ad-a24605ff3170-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:58:54.866675 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.866651    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr6ps\" (UniqueName: \"kubernetes.io/projected/351285ca-be57-4fa2-91ad-a24605ff3170-kube-api-access-fr6ps\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:58:54.872955 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.872930    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"]
Sep 16 13:58:54.879096 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:54.879075    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d6baf3-7644859c65-b7tkj"]
Sep 16 13:58:56.803118 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:56.803084    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="351285ca-be57-4fa2-91ad-a24605ff3170" path="/var/lib/kubelet/pods/351285ca-be57-4fa2-91ad-a24605ff3170/volumes"
Sep 16 13:58:59.846453 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.846423    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"]
Sep 16 13:58:59.846822 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.846771    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351285ca-be57-4fa2-91ad-a24605ff3170" containerName="payload-processing"
Sep 16 13:58:59.846822 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.846783    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="351285ca-be57-4fa2-91ad-a24605ff3170" containerName="payload-processing"
Sep 16 13:58:59.846822 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.846797    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4b0547-2d4e-4c79-8b71-989ef2040dad" containerName="revoke-keys"
Sep 16 13:58:59.846822 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.846802    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4b0547-2d4e-4c79-8b71-989ef2040dad" containerName="revoke-keys"
Sep 16 13:58:59.846975 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.846868    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="351285ca-be57-4fa2-91ad-a24605ff3170" containerName="payload-processing"
Sep 16 13:58:59.846975 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.846877    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4b0547-2d4e-4c79-8b71-989ef2040dad" containerName="revoke-keys"
Sep 16 13:58:59.851376 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.851359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"
Sep 16 13:58:59.853423 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.853402    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:58:59.853548 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.853403    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:58:59.856430 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.856409    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"]
Sep 16 13:58:59.909415 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:58:59.909377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d51a6927-7544-47d9-ab3b-eaa0997a8d71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh\" (UID: \"d51a6927-7544-47d9-ab3b-eaa0997a8d71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"
Sep 16 13:59:00.010049 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:00.010022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d51a6927-7544-47d9-ab3b-eaa0997a8d71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh\" (UID: \"d51a6927-7544-47d9-ab3b-eaa0997a8d71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"
Sep 16 13:59:00.010812 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:00.010789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d51a6927-7544-47d9-ab3b-eaa0997a8d71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh\" (UID: \"d51a6927-7544-47d9-ab3b-eaa0997a8d71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"
Sep 16 13:59:00.163623 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:00.163594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"
Sep 16 13:59:00.432477 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:00.432310    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"]
Sep 16 13:59:00.435206 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:59:00.435179    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd51a6927_7544_47d9_ab3b_eaa0997a8d71.slice/crio-04768e0ee6e66972cd68de1dd2876d37af51d6079bade5c3e9a1973f445e9330 WatchSource:0}: Error finding container 04768e0ee6e66972cd68de1dd2876d37af51d6079bade5c3e9a1973f445e9330: Status 404 returned error can't find the container with id 04768e0ee6e66972cd68de1dd2876d37af51d6079bade5c3e9a1973f445e9330
Sep 16 13:59:00.889076 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:00.889042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh" event={"ID":"d51a6927-7544-47d9-ab3b-eaa0997a8d71","Type":"ContainerStarted","Data":"9a7dd8182a0986c4fb20ca0f537a54234bcee7ef5ad036a66574fc4bf13cf244"}
Sep 16 13:59:00.889472 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:00.889094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh" event={"ID":"d51a6927-7544-47d9-ab3b-eaa0997a8d71","Type":"ContainerStarted","Data":"04768e0ee6e66972cd68de1dd2876d37af51d6079bade5c3e9a1973f445e9330"}
Sep 16 13:59:00.903574 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:00.903479    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh" podStartSLOduration=1.9034611369999999 podStartE2EDuration="1.903461137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:59:00.903167088 +0000 UTC m=+1568.586825496" watchObservedRunningTime="2026-09-16 13:59:00.903461137 +0000 UTC m=+1568.587119523"
Sep 16 13:59:01.859995 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:01.859961    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"]
Sep 16 13:59:01.863273 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:01.863252    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"
Sep 16 13:59:01.871923 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:01.871883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"]
Sep 16 13:59:01.897382 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:01.897348    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51a6927-7544-47d9-ab3b-eaa0997a8d71" containerID="9a7dd8182a0986c4fb20ca0f537a54234bcee7ef5ad036a66574fc4bf13cf244" exitCode=0
Sep 16 13:59:01.897974 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:01.897522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh" event={"ID":"d51a6927-7544-47d9-ab3b-eaa0997a8d71","Type":"ContainerDied","Data":"9a7dd8182a0986c4fb20ca0f537a54234bcee7ef5ad036a66574fc4bf13cf244"}
Sep 16 13:59:01.927342 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:01.927311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g\" (UID: \"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"
Sep 16 13:59:02.028000 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:02.027962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g\" (UID: \"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"
Sep 16 13:59:02.028552 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:02.028531    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g\" (UID: \"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"
Sep 16 13:59:02.174971 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:02.174932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"
Sep 16 13:59:02.462050 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:02.461965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"]
Sep 16 13:59:02.465005 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:59:02.464971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d04d8c3_6e38_4582_bbb8_f9d293d9a5ae.slice/crio-719cbcad2032df801ef3ed86aca6bf82c58920c1178da9cd592397992f205741 WatchSource:0}: Error finding container 719cbcad2032df801ef3ed86aca6bf82c58920c1178da9cd592397992f205741: Status 404 returned error can't find the container with id 719cbcad2032df801ef3ed86aca6bf82c58920c1178da9cd592397992f205741
Sep 16 13:59:02.904258 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:02.904227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g" event={"ID":"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae","Type":"ContainerStarted","Data":"a4c504c8e331bd41ffa73e8f7fa2cd96fc5f2c03282e7ac6f7e212c7fc986b64"}
Sep 16 13:59:02.904698 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:02.904282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g" event={"ID":"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae","Type":"ContainerStarted","Data":"719cbcad2032df801ef3ed86aca6bf82c58920c1178da9cd592397992f205741"}
Sep 16 13:59:02.916681 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:02.916548    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g" podStartSLOduration=1.916528922 podStartE2EDuration="1.916528922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:59:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:59:02.916133143 +0000 UTC m=+1570.599791536" watchObservedRunningTime="2026-09-16 13:59:02.916528922 +0000 UTC m=+1570.600187320"
Sep 16 13:59:03.024569 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.024546    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"
Sep 16 13:59:03.141219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.141184    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d51a6927-7544-47d9-ab3b-eaa0997a8d71-maas-api-service-ca\") pod \"d51a6927-7544-47d9-ab3b-eaa0997a8d71\" (UID: \"d51a6927-7544-47d9-ab3b-eaa0997a8d71\") "
Sep 16 13:59:03.141511 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.141489    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51a6927-7544-47d9-ab3b-eaa0997a8d71-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d51a6927-7544-47d9-ab3b-eaa0997a8d71" (UID: "d51a6927-7544-47d9-ab3b-eaa0997a8d71"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:03.242560 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.242530    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d51a6927-7544-47d9-ab3b-eaa0997a8d71-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:03.910829 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.910790    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae" containerID="a4c504c8e331bd41ffa73e8f7fa2cd96fc5f2c03282e7ac6f7e212c7fc986b64" exitCode=0
Sep 16 13:59:03.911298 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.910840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g" event={"ID":"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae","Type":"ContainerDied","Data":"a4c504c8e331bd41ffa73e8f7fa2cd96fc5f2c03282e7ac6f7e212c7fc986b64"}
Sep 16 13:59:03.912289 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.912269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"
Sep 16 13:59:03.912396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.912293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh" event={"ID":"d51a6927-7544-47d9-ab3b-eaa0997a8d71","Type":"ContainerDied","Data":"04768e0ee6e66972cd68de1dd2876d37af51d6079bade5c3e9a1973f445e9330"}
Sep 16 13:59:03.912396 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:03.912332    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04768e0ee6e66972cd68de1dd2876d37af51d6079bade5c3e9a1973f445e9330"
Sep 16 13:59:04.738432 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:04.738395    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"]
Sep 16 13:59:05.052470 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:05.052447    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"
Sep 16 13:59:05.158894 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:05.158864    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae-maas-api-service-ca\") pod \"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae\" (UID: \"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae\") "
Sep 16 13:59:05.159200 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:05.159177    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae" (UID: "3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:05.259973 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:05.259932    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:05.923381 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:05.923352    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"
Sep 16 13:59:05.923381 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:05.923378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g" event={"ID":"3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae","Type":"ContainerDied","Data":"719cbcad2032df801ef3ed86aca6bf82c58920c1178da9cd592397992f205741"}
Sep 16 13:59:05.923589 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:05.923443    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="719cbcad2032df801ef3ed86aca6bf82c58920c1178da9cd592397992f205741"
Sep 16 13:59:12.760933 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:12.755679    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"]
Sep 16 13:59:12.760933 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:12.756081    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl" podUID="4cb632d5-4bf5-4af9-a7eb-26c6424a3750" containerName="payload-processing" containerID="cri-o://09ed1d216fb972603c71745d0af4894041fb5b5eab202e37f6a3e7cb8765896b" gracePeriod=30
Sep 16 13:59:12.916419 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:12.916381    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"]
Sep 16 13:59:12.919884 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:12.919856    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4fa336-jfb7g"]
Sep 16 13:59:12.960174 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:12.960139    2580 generic.go:356] "Generic (PLEG): container finished" podID="4cb632d5-4bf5-4af9-a7eb-26c6424a3750" containerID="09ed1d216fb972603c71745d0af4894041fb5b5eab202e37f6a3e7cb8765896b" exitCode=0
Sep 16 13:59:12.960323 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:12.960217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl" event={"ID":"4cb632d5-4bf5-4af9-a7eb-26c6424a3750","Type":"ContainerDied","Data":"09ed1d216fb972603c71745d0af4894041fb5b5eab202e37f6a3e7cb8765896b"}
Sep 16 13:59:13.017512 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.017450    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:59:13.138304 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.138275    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-plugins-config-volume\") pod \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") "
Sep 16 13:59:13.138492 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.138316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j57bt\" (UniqueName: \"kubernetes.io/projected/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-kube-api-access-j57bt\") pod \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\" (UID: \"4cb632d5-4bf5-4af9-a7eb-26c6424a3750\") "
Sep 16 13:59:13.138700 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.138664    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4cb632d5-4bf5-4af9-a7eb-26c6424a3750" (UID: "4cb632d5-4bf5-4af9-a7eb-26c6424a3750"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:13.140380 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.140355    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-kube-api-access-j57bt" (OuterVolumeSpecName: "kube-api-access-j57bt") pod "4cb632d5-4bf5-4af9-a7eb-26c6424a3750" (UID: "4cb632d5-4bf5-4af9-a7eb-26c6424a3750"). InnerVolumeSpecName "kube-api-access-j57bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:59:13.239886 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.239853    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:13.239886 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.239884    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j57bt\" (UniqueName: \"kubernetes.io/projected/4cb632d5-4bf5-4af9-a7eb-26c6424a3750-kube-api-access-j57bt\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:13.966185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.966152    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"
Sep 16 13:59:13.966600 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.966189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl" event={"ID":"4cb632d5-4bf5-4af9-a7eb-26c6424a3750","Type":"ContainerDied","Data":"de2bcce3bf5768267e8e390f39e21728153e651fa8ce2b8a4ffd8ce8f9e62f27"}
Sep 16 13:59:13.966600 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.966249    2580 scope.go:117] "RemoveContainer" containerID="09ed1d216fb972603c71745d0af4894041fb5b5eab202e37f6a3e7cb8765896b"
Sep 16 13:59:13.985935 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.985878    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"]
Sep 16 13:59:13.989024 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:13.989004    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4fa336-557b4df4f5-tzksl"]
Sep 16 13:59:14.802974 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:14.802940    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae" path="/var/lib/kubelet/pods/3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae/volumes"
Sep 16 13:59:14.803311 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:14.803298    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cb632d5-4bf5-4af9-a7eb-26c6424a3750" path="/var/lib/kubelet/pods/4cb632d5-4bf5-4af9-a7eb-26c6424a3750/volumes"
Sep 16 13:59:15.478737 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.478701    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"]
Sep 16 13:59:15.479124 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.478965    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r" podUID="4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" containerName="payload-processing" containerID="cri-o://39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f" gracePeriod=30
Sep 16 13:59:15.485319 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.485295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"]
Sep 16 13:59:15.485543 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.485515    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c" podUID="f3fc5ef3-057c-41fb-bec4-2a70ca89f058" containerName="payload-pre-processing" containerID="cri-o://798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f" gracePeriod=30
Sep 16 13:59:15.723065 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.723025    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"]
Sep 16 13:59:15.725599 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.725577    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fc8c65-mtsgh"]
Sep 16 13:59:15.741938 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.741876    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:59:15.753359 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.753342    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:59:15.864300 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.864271    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-plugins-config-volume\") pod \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") "
Sep 16 13:59:15.864436 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.864346    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45p4k\" (UniqueName: \"kubernetes.io/projected/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-kube-api-access-45p4k\") pod \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") "
Sep 16 13:59:15.864436 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.864368    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-plugins-config-volume\") pod \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\" (UID: \"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab\") "
Sep 16 13:59:15.864436 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.864401    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcbfr\" (UniqueName: \"kubernetes.io/projected/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-kube-api-access-mcbfr\") pod \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\" (UID: \"f3fc5ef3-057c-41fb-bec4-2a70ca89f058\") "
Sep 16 13:59:15.864703 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.864669    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3fc5ef3-057c-41fb-bec4-2a70ca89f058" (UID: "f3fc5ef3-057c-41fb-bec4-2a70ca89f058"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:15.864820 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.864740    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" (UID: "4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:15.866419 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.866395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-kube-api-access-mcbfr" (OuterVolumeSpecName: "kube-api-access-mcbfr") pod "f3fc5ef3-057c-41fb-bec4-2a70ca89f058" (UID: "f3fc5ef3-057c-41fb-bec4-2a70ca89f058"). InnerVolumeSpecName "kube-api-access-mcbfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:59:15.866501 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.866462    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-kube-api-access-45p4k" (OuterVolumeSpecName: "kube-api-access-45p4k") pod "4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" (UID: "4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab"). InnerVolumeSpecName "kube-api-access-45p4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:59:15.965178 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.965152    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:15.965178 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.965174    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45p4k\" (UniqueName: \"kubernetes.io/projected/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-kube-api-access-45p4k\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:15.965326 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.965186    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:15.965326 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.965194    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcbfr\" (UniqueName: \"kubernetes.io/projected/f3fc5ef3-057c-41fb-bec4-2a70ca89f058-kube-api-access-mcbfr\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:15.977833 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.977807    2580 generic.go:356] "Generic (PLEG): container finished" podID="f3fc5ef3-057c-41fb-bec4-2a70ca89f058" containerID="798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f" exitCode=0
Sep 16 13:59:15.977967 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.977854    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"
Sep 16 13:59:15.977967 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.977919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c" event={"ID":"f3fc5ef3-057c-41fb-bec4-2a70ca89f058","Type":"ContainerDied","Data":"798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f"}
Sep 16 13:59:15.978066 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.977978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c" event={"ID":"f3fc5ef3-057c-41fb-bec4-2a70ca89f058","Type":"ContainerDied","Data":"7d6e65a593eed8898035928e383e4511357d36c92ee83d4981b409793b71bd97"}
Sep 16 13:59:15.978066 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.978009    2580 scope.go:117] "RemoveContainer" containerID="798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f"
Sep 16 13:59:15.979517 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.979494    2580 generic.go:356] "Generic (PLEG): container finished" podID="4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" containerID="39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f" exitCode=0
Sep 16 13:59:15.979620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.979528    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"
Sep 16 13:59:15.979620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.979531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r" event={"ID":"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab","Type":"ContainerDied","Data":"39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f"}
Sep 16 13:59:15.979620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.979568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r" event={"ID":"4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab","Type":"ContainerDied","Data":"d400aeeb899af08760d366fa3a117454d348bf48ec54550cd73715c3673a1885"}
Sep 16 13:59:15.988116 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.987944    2580 scope.go:117] "RemoveContainer" containerID="798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f"
Sep 16 13:59:15.988314 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:59:15.988286    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f\": container with ID starting with 798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f not found: ID does not exist" containerID="798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f"
Sep 16 13:59:15.988383 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.988363    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f"} err="failed to get container status \"798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f\": rpc error: code = NotFound desc = could not find container \"798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f\": container with ID starting with 798e3745e3820c117095551d666546da359b5e90a0420fdf58acf13beb91992f not found: ID does not exist"
Sep 16 13:59:15.988440 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.988385    2580 scope.go:117] "RemoveContainer" containerID="39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f"
Sep 16 13:59:15.996656 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.996636    2580 scope.go:117] "RemoveContainer" containerID="39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f"
Sep 16 13:59:15.996891 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:59:15.996875    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f\": container with ID starting with 39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f not found: ID does not exist" containerID="39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f"
Sep 16 13:59:15.996948 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:15.996912    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f"} err="failed to get container status \"39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f\": rpc error: code = NotFound desc = could not find container \"39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f\": container with ID starting with 39796368deab5362a5c885cdca68ed2916880ad0c36bd401f430829577ca612f not found: ID does not exist"
Sep 16 13:59:16.007169 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:16.007150    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"]
Sep 16 13:59:16.014840 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:16.014820    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-fc8c65-567d4d5f45-rd49r"]
Sep 16 13:59:16.030336 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:16.030312    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"]
Sep 16 13:59:16.039989 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:16.039968    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-fc8c65-557cf9fd78-8ln2c"]
Sep 16 13:59:16.803038 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:16.802960    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" path="/var/lib/kubelet/pods/4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab/volumes"
Sep 16 13:59:16.803377 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:16.803289    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d51a6927-7544-47d9-ab3b-eaa0997a8d71" path="/var/lib/kubelet/pods/d51a6927-7544-47d9-ab3b-eaa0997a8d71/volumes"
Sep 16 13:59:16.803594 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:16.803581    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3fc5ef3-057c-41fb-bec4-2a70ca89f058" path="/var/lib/kubelet/pods/f3fc5ef3-057c-41fb-bec4-2a70ca89f058/volumes"
Sep 16 13:59:19.757306 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:19.757266    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw" podUID="3c8c4444-da71-4439-9604-b55292be7cb1" containerName="main" containerID="cri-o://174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5" gracePeriod=45
Sep 16 13:59:19.998880 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:19.998854    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:59:20.006960 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.006936    2580 generic.go:356] "Generic (PLEG): container finished" podID="3c8c4444-da71-4439-9604-b55292be7cb1" containerID="174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5" exitCode=0
Sep 16 13:59:20.007062 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.006976    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"
Sep 16 13:59:20.007062 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.007022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw" event={"ID":"3c8c4444-da71-4439-9604-b55292be7cb1","Type":"ContainerDied","Data":"174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5"}
Sep 16 13:59:20.007160 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.007064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw" event={"ID":"3c8c4444-da71-4439-9604-b55292be7cb1","Type":"ContainerDied","Data":"0c682c42be0fae0bf0d19d8fde619ca8e498845cbd79d194f5c82c036aec9fe9"}
Sep 16 13:59:20.007160 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.007089    2580 scope.go:117] "RemoveContainer" containerID="174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5"
Sep 16 13:59:20.015221 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.015201    2580 scope.go:117] "RemoveContainer" containerID="174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5"
Sep 16 13:59:20.015527 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:59:20.015503    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5\": container with ID starting with 174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5 not found: ID does not exist" containerID="174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5"
Sep 16 13:59:20.015633 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.015537    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5"} err="failed to get container status \"174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5\": rpc error: code = NotFound desc = could not find container \"174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5\": container with ID starting with 174460167dcec19f65dbfda3345d5fdb3863ab5645cb8a6d4cf63458cba2cfb5 not found: ID does not exist"
Sep 16 13:59:20.098366 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098335    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-tmp-dir\") pod \"3c8c4444-da71-4439-9604-b55292be7cb1\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") "
Sep 16 13:59:20.098767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098381    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-home\") pod \"3c8c4444-da71-4439-9604-b55292be7cb1\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") "
Sep 16 13:59:20.098767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scjjm\" (UniqueName: \"kubernetes.io/projected/3c8c4444-da71-4439-9604-b55292be7cb1-kube-api-access-scjjm\") pod \"3c8c4444-da71-4439-9604-b55292be7cb1\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") "
Sep 16 13:59:20.098767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098441    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-model-cache\") pod \"3c8c4444-da71-4439-9604-b55292be7cb1\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") "
Sep 16 13:59:20.098767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098455    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-dshm\") pod \"3c8c4444-da71-4439-9604-b55292be7cb1\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") "
Sep 16 13:59:20.098767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8c4444-da71-4439-9604-b55292be7cb1-tls-certs\") pod \"3c8c4444-da71-4439-9604-b55292be7cb1\" (UID: \"3c8c4444-da71-4439-9604-b55292be7cb1\") "
Sep 16 13:59:20.098767 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098727    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-home" (OuterVolumeSpecName: "home") pod "3c8c4444-da71-4439-9604-b55292be7cb1" (UID: "3c8c4444-da71-4439-9604-b55292be7cb1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:20.099185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098836    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c8c4444-da71-4439-9604-b55292be7cb1" (UID: "3c8c4444-da71-4439-9604-b55292be7cb1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:20.099185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.098940    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c8c4444-da71-4439-9604-b55292be7cb1" (UID: "3c8c4444-da71-4439-9604-b55292be7cb1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:20.099185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.099077    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:20.099185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.099090    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:20.099185 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.099100    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:20.101015 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.100989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c8c4444-da71-4439-9604-b55292be7cb1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c8c4444-da71-4439-9604-b55292be7cb1" (UID: "3c8c4444-da71-4439-9604-b55292be7cb1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:59:20.101131 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.101089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c8c4444-da71-4439-9604-b55292be7cb1-kube-api-access-scjjm" (OuterVolumeSpecName: "kube-api-access-scjjm") pod "3c8c4444-da71-4439-9604-b55292be7cb1" (UID: "3c8c4444-da71-4439-9604-b55292be7cb1"). InnerVolumeSpecName "kube-api-access-scjjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:59:20.101204 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.101176    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-dshm" (OuterVolumeSpecName: "dshm") pod "3c8c4444-da71-4439-9604-b55292be7cb1" (UID: "3c8c4444-da71-4439-9604-b55292be7cb1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:20.164864 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.164827    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"]
Sep 16 13:59:20.165179 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165166    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51a6927-7544-47d9-ab3b-eaa0997a8d71" containerName="revoke-keys"
Sep 16 13:59:20.165219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165180    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51a6927-7544-47d9-ab3b-eaa0997a8d71" containerName="revoke-keys"
Sep 16 13:59:20.165219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165191    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae" containerName="revoke-keys"
Sep 16 13:59:20.165219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165196    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae" containerName="revoke-keys"
Sep 16 13:59:20.165219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165204    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c8c4444-da71-4439-9604-b55292be7cb1" containerName="main"
Sep 16 13:59:20.165219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165210    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c8c4444-da71-4439-9604-b55292be7cb1" containerName="main"
Sep 16 13:59:20.165219 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165217    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fc5ef3-057c-41fb-bec4-2a70ca89f058" containerName="payload-pre-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165223    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fc5ef3-057c-41fb-bec4-2a70ca89f058" containerName="payload-pre-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165230    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" containerName="payload-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165235    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" containerName="payload-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165242    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb632d5-4bf5-4af9-a7eb-26c6424a3750" containerName="payload-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165247    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb632d5-4bf5-4af9-a7eb-26c6424a3750" containerName="payload-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165292    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cb632d5-4bf5-4af9-a7eb-26c6424a3750" containerName="payload-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165301    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51a6927-7544-47d9-ab3b-eaa0997a8d71" containerName="revoke-keys"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165307    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d04d8c3-6e38-4582-bbb8-f9d293d9a5ae" containerName="revoke-keys"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165313    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c8c4444-da71-4439-9604-b55292be7cb1" containerName="main"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165319    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c70740f-a09c-42c9-a0c3-4ced6bf3d5ab" containerName="payload-processing"
Sep 16 13:59:20.165416 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.165326    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fc5ef3-057c-41fb-bec4-2a70ca89f058" containerName="payload-pre-processing"
Sep 16 13:59:20.169756 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.169738    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"
Sep 16 13:59:20.171720 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.171697    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 13:59:20.171907 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.171882    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 13:59:20.175218 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.175195    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"]
Sep 16 13:59:20.200451 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.200428    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scjjm\" (UniqueName: \"kubernetes.io/projected/3c8c4444-da71-4439-9604-b55292be7cb1-kube-api-access-scjjm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:20.200451 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.200450    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c8c4444-da71-4439-9604-b55292be7cb1-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:20.200627 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.200459    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8c4444-da71-4439-9604-b55292be7cb1-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:20.301390 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.301297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/045e2ee0-b9d2-42df-a829-2a9727fefdb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q\" (UID: \"045e2ee0-b9d2-42df-a829-2a9727fefdb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"
Sep 16 13:59:20.329301 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.329277    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"]
Sep 16 13:59:20.333490 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.333461    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-fc8c65/premium-w4-fc8c65-kserve-69f6d9cbfc-bwqnw"]
Sep 16 13:59:20.402539 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.402515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/045e2ee0-b9d2-42df-a829-2a9727fefdb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q\" (UID: \"045e2ee0-b9d2-42df-a829-2a9727fefdb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"
Sep 16 13:59:20.403099 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.403080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/045e2ee0-b9d2-42df-a829-2a9727fefdb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q\" (UID: \"045e2ee0-b9d2-42df-a829-2a9727fefdb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"
Sep 16 13:59:20.481633 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.481609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"
Sep 16 13:59:20.751447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.751278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"]
Sep 16 13:59:20.753991 ip-10-0-130-207 kubenswrapper[2580]: W0916 13:59:20.753964    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045e2ee0_b9d2_42df_a829_2a9727fefdb1.slice/crio-0243da71905ec3d3e719b5e1427b00698820c26e719cbbf55d865321cb1f7f43 WatchSource:0}: Error finding container 0243da71905ec3d3e719b5e1427b00698820c26e719cbbf55d865321cb1f7f43: Status 404 returned error can't find the container with id 0243da71905ec3d3e719b5e1427b00698820c26e719cbbf55d865321cb1f7f43
Sep 16 13:59:20.802601 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:20.802572    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c8c4444-da71-4439-9604-b55292be7cb1" path="/var/lib/kubelet/pods/3c8c4444-da71-4439-9604-b55292be7cb1/volumes"
Sep 16 13:59:21.014514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:21.014481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q" event={"ID":"045e2ee0-b9d2-42df-a829-2a9727fefdb1","Type":"ContainerStarted","Data":"5ac3a07ec90fdac7b15c9d8953098946000aaa2807d6601e6a0d0f95618560f8"}
Sep 16 13:59:21.014679 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:21.014571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q" event={"ID":"045e2ee0-b9d2-42df-a829-2a9727fefdb1","Type":"ContainerStarted","Data":"0243da71905ec3d3e719b5e1427b00698820c26e719cbbf55d865321cb1f7f43"}
Sep 16 13:59:21.029800 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:21.029756    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q" podStartSLOduration=1.029740693 podStartE2EDuration="1.029740693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 13:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 13:59:21.028091455 +0000 UTC m=+1588.711749850" watchObservedRunningTime="2026-09-16 13:59:21.029740693 +0000 UTC m=+1588.713399084"
Sep 16 13:59:22.020333 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:22.020296    2580 generic.go:356] "Generic (PLEG): container finished" podID="045e2ee0-b9d2-42df-a829-2a9727fefdb1" containerID="5ac3a07ec90fdac7b15c9d8953098946000aaa2807d6601e6a0d0f95618560f8" exitCode=0
Sep 16 13:59:22.020697 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:22.020368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q" event={"ID":"045e2ee0-b9d2-42df-a829-2a9727fefdb1","Type":"ContainerDied","Data":"5ac3a07ec90fdac7b15c9d8953098946000aaa2807d6601e6a0d0f95618560f8"}
Sep 16 13:59:23.159425 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:23.159399    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"
Sep 16 13:59:23.325596 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:23.325514    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/045e2ee0-b9d2-42df-a829-2a9727fefdb1-maas-api-service-ca\") pod \"045e2ee0-b9d2-42df-a829-2a9727fefdb1\" (UID: \"045e2ee0-b9d2-42df-a829-2a9727fefdb1\") "
Sep 16 13:59:23.325850 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:23.325825    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/045e2ee0-b9d2-42df-a829-2a9727fefdb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "045e2ee0-b9d2-42df-a829-2a9727fefdb1" (UID: "045e2ee0-b9d2-42df-a829-2a9727fefdb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:23.427049 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:23.427022    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/045e2ee0-b9d2-42df-a829-2a9727fefdb1-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:24.031164 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:24.031136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"
Sep 16 13:59:24.031164 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:24.031150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q" event={"ID":"045e2ee0-b9d2-42df-a829-2a9727fefdb1","Type":"ContainerDied","Data":"0243da71905ec3d3e719b5e1427b00698820c26e719cbbf55d865321cb1f7f43"}
Sep 16 13:59:24.031744 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:24.031195    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0243da71905ec3d3e719b5e1427b00698820c26e719cbbf55d865321cb1f7f43"
Sep 16 13:59:30.668210 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.668173    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"]
Sep 16 13:59:30.668613 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.668431    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj" podUID="ef73f2a4-84ec-4b34-acca-133473411a1a" containerName="payload-processing" containerID="cri-o://e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77" gracePeriod=30
Sep 16 13:59:30.678266 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.678242    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"]
Sep 16 13:59:30.678440 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.678420    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv" podUID="ea97e996-a2fe-4c6b-81c1-766645456f5c" containerName="payload-pre-processing" containerID="cri-o://c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c" gracePeriod=30
Sep 16 13:59:30.840501 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.840415    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"]
Sep 16 13:59:30.844768 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.844738    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-284b60-whf9q"]
Sep 16 13:59:30.920058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.919999    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:59:30.949020 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:30.948993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:59:31.069965 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.069932    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea97e996-a2fe-4c6b-81c1-766645456f5c" containerID="c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c" exitCode=0
Sep 16 13:59:31.069965 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.069964    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"
Sep 16 13:59:31.070213 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.070016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv" event={"ID":"ea97e996-a2fe-4c6b-81c1-766645456f5c","Type":"ContainerDied","Data":"c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c"}
Sep 16 13:59:31.070213 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.070061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv" event={"ID":"ea97e996-a2fe-4c6b-81c1-766645456f5c","Type":"ContainerDied","Data":"1a99288a54c95a4d3975c9c5d3e26eeadded1e94a88099dd2646c78f57c5025f"}
Sep 16 13:59:31.070213 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.070084    2580 scope.go:117] "RemoveContainer" containerID="c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c"
Sep 16 13:59:31.071325 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.071298    2580 generic.go:356] "Generic (PLEG): container finished" podID="ef73f2a4-84ec-4b34-acca-133473411a1a" containerID="e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77" exitCode=0
Sep 16 13:59:31.071426 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.071324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj" event={"ID":"ef73f2a4-84ec-4b34-acca-133473411a1a","Type":"ContainerDied","Data":"e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77"}
Sep 16 13:59:31.071426 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.071343    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"
Sep 16 13:59:31.071426 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.071370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj" event={"ID":"ef73f2a4-84ec-4b34-acca-133473411a1a","Type":"ContainerDied","Data":"bd74fc14016198f1032bc819893522654451317db04e26a2fb39fd587b9ede1f"}
Sep 16 13:59:31.078337 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.078315    2580 scope.go:117] "RemoveContainer" containerID="c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c"
Sep 16 13:59:31.078592 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:59:31.078573    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c\": container with ID starting with c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c not found: ID does not exist" containerID="c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c"
Sep 16 13:59:31.078663 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.078601    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c"} err="failed to get container status \"c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c\": rpc error: code = NotFound desc = could not find container \"c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c\": container with ID starting with c839103d611ac81f463ea208d77dfb503dcb6468929dea984160b4e0ac36d62c not found: ID does not exist"
Sep 16 13:59:31.078663 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.078623    2580 scope.go:117] "RemoveContainer" containerID="e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77"
Sep 16 13:59:31.086015 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.086001    2580 scope.go:117] "RemoveContainer" containerID="e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77"
Sep 16 13:59:31.086293 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:59:31.086270    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77\": container with ID starting with e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77 not found: ID does not exist" containerID="e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77"
Sep 16 13:59:31.086348 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.086292    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77"} err="failed to get container status \"e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77\": rpc error: code = NotFound desc = could not find container \"e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77\": container with ID starting with e7f6b06e2dc5fdc408af06bfc81ad7c1604096e8629ac7aabe0360f621739e77 not found: ID does not exist"
Sep 16 13:59:31.091521 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.091501    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzh42\" (UniqueName: \"kubernetes.io/projected/ea97e996-a2fe-4c6b-81c1-766645456f5c-kube-api-access-fzh42\") pod \"ea97e996-a2fe-4c6b-81c1-766645456f5c\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") "
Sep 16 13:59:31.091620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.091549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef73f2a4-84ec-4b34-acca-133473411a1a-plugins-config-volume\") pod \"ef73f2a4-84ec-4b34-acca-133473411a1a\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") "
Sep 16 13:59:31.091620 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.091610    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea97e996-a2fe-4c6b-81c1-766645456f5c-plugins-config-volume\") pod \"ea97e996-a2fe-4c6b-81c1-766645456f5c\" (UID: \"ea97e996-a2fe-4c6b-81c1-766645456f5c\") "
Sep 16 13:59:31.091710 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.091647    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbhpk\" (UniqueName: \"kubernetes.io/projected/ef73f2a4-84ec-4b34-acca-133473411a1a-kube-api-access-wbhpk\") pod \"ef73f2a4-84ec-4b34-acca-133473411a1a\" (UID: \"ef73f2a4-84ec-4b34-acca-133473411a1a\") "
Sep 16 13:59:31.092010 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.091985    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef73f2a4-84ec-4b34-acca-133473411a1a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef73f2a4-84ec-4b34-acca-133473411a1a" (UID: "ef73f2a4-84ec-4b34-acca-133473411a1a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:31.092119 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.092023    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea97e996-a2fe-4c6b-81c1-766645456f5c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea97e996-a2fe-4c6b-81c1-766645456f5c" (UID: "ea97e996-a2fe-4c6b-81c1-766645456f5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 13:59:31.093658 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.093634    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea97e996-a2fe-4c6b-81c1-766645456f5c-kube-api-access-fzh42" (OuterVolumeSpecName: "kube-api-access-fzh42") pod "ea97e996-a2fe-4c6b-81c1-766645456f5c" (UID: "ea97e996-a2fe-4c6b-81c1-766645456f5c"). InnerVolumeSpecName "kube-api-access-fzh42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:59:31.093658 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.093645    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef73f2a4-84ec-4b34-acca-133473411a1a-kube-api-access-wbhpk" (OuterVolumeSpecName: "kube-api-access-wbhpk") pod "ef73f2a4-84ec-4b34-acca-133473411a1a" (UID: "ef73f2a4-84ec-4b34-acca-133473411a1a"). InnerVolumeSpecName "kube-api-access-wbhpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:59:31.192687 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.192604    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzh42\" (UniqueName: \"kubernetes.io/projected/ea97e996-a2fe-4c6b-81c1-766645456f5c-kube-api-access-fzh42\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:31.192687 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.192634    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef73f2a4-84ec-4b34-acca-133473411a1a-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:31.192687 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.192644    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea97e996-a2fe-4c6b-81c1-766645456f5c-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:31.192687 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.192653    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbhpk\" (UniqueName: \"kubernetes.io/projected/ef73f2a4-84ec-4b34-acca-133473411a1a-kube-api-access-wbhpk\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:31.393351 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.393320    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"]
Sep 16 13:59:31.395611 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.395588    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-284b60-6dfb4bcc66-26tpv"]
Sep 16 13:59:31.404570 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.404545    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"]
Sep 16 13:59:31.408039 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:31.408018    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-284b60-5979976d4c-pnttj"]
Sep 16 13:59:32.802745 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:32.802711    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="045e2ee0-b9d2-42df-a829-2a9727fefdb1" path="/var/lib/kubelet/pods/045e2ee0-b9d2-42df-a829-2a9727fefdb1/volumes"
Sep 16 13:59:32.803120 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:32.803072    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea97e996-a2fe-4c6b-81c1-766645456f5c" path="/var/lib/kubelet/pods/ea97e996-a2fe-4c6b-81c1-766645456f5c/volumes"
Sep 16 13:59:32.803378 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:32.803365    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef73f2a4-84ec-4b34-acca-133473411a1a" path="/var/lib/kubelet/pods/ef73f2a4-84ec-4b34-acca-133473411a1a/volumes"
Sep 16 13:59:40.690268 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:40.690231    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"]
Sep 16 13:59:55.710013 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.709943    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5" podUID="a9f046f0-d92b-49c3-bfe0-7bb86208079b" containerName="main" containerID="cri-o://d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24" gracePeriod=45
Sep 16 13:59:55.945797 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.945772    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:59:55.994756 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.994675    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-dshm\") pod \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") "
Sep 16 13:59:55.994756 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.994714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-home\") pod \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") "
Sep 16 13:59:55.994756 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.994759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tmp-dir\") pod \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") "
Sep 16 13:59:55.995112 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.994778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-model-cache\") pod \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") "
Sep 16 13:59:55.995112 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.994810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tls-certs\") pod \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") "
Sep 16 13:59:55.995112 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.994862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcrwl\" (UniqueName: \"kubernetes.io/projected/a9f046f0-d92b-49c3-bfe0-7bb86208079b-kube-api-access-kcrwl\") pod \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\" (UID: \"a9f046f0-d92b-49c3-bfe0-7bb86208079b\") "
Sep 16 13:59:55.995112 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.995031    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a9f046f0-d92b-49c3-bfe0-7bb86208079b" (UID: "a9f046f0-d92b-49c3-bfe0-7bb86208079b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:55.995112 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.995095    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-home" (OuterVolumeSpecName: "home") pod "a9f046f0-d92b-49c3-bfe0-7bb86208079b" (UID: "a9f046f0-d92b-49c3-bfe0-7bb86208079b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:55.995334 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.995109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a9f046f0-d92b-49c3-bfe0-7bb86208079b" (UID: "a9f046f0-d92b-49c3-bfe0-7bb86208079b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:55.995334 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.995151    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:55.995334 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.995163    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:55.996944 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.996916    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-dshm" (OuterVolumeSpecName: "dshm") pod "a9f046f0-d92b-49c3-bfe0-7bb86208079b" (UID: "a9f046f0-d92b-49c3-bfe0-7bb86208079b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 13:59:55.997057 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.996965    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a9f046f0-d92b-49c3-bfe0-7bb86208079b" (UID: "a9f046f0-d92b-49c3-bfe0-7bb86208079b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 13:59:55.997057 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:55.997017    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f046f0-d92b-49c3-bfe0-7bb86208079b-kube-api-access-kcrwl" (OuterVolumeSpecName: "kube-api-access-kcrwl") pod "a9f046f0-d92b-49c3-bfe0-7bb86208079b" (UID: "a9f046f0-d92b-49c3-bfe0-7bb86208079b"). InnerVolumeSpecName "kube-api-access-kcrwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 13:59:56.096058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.096020    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f046f0-d92b-49c3-bfe0-7bb86208079b-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:56.096058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.096050    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcrwl\" (UniqueName: \"kubernetes.io/projected/a9f046f0-d92b-49c3-bfe0-7bb86208079b-kube-api-access-kcrwl\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:56.096058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.096060    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:56.096058 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.096068    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9f046f0-d92b-49c3-bfe0-7bb86208079b-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 13:59:56.191266 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.191231    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9f046f0-d92b-49c3-bfe0-7bb86208079b" containerID="d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24" exitCode=0
Sep 16 13:59:56.191447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.191278    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"
Sep 16 13:59:56.191447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.191313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5" event={"ID":"a9f046f0-d92b-49c3-bfe0-7bb86208079b","Type":"ContainerDied","Data":"d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24"}
Sep 16 13:59:56.191447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.191357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5" event={"ID":"a9f046f0-d92b-49c3-bfe0-7bb86208079b","Type":"ContainerDied","Data":"c967dbf04ca00a49514d028e49bf82e1798861040686d69a5317a47a00573c8b"}
Sep 16 13:59:56.191447 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.191374    2580 scope.go:117] "RemoveContainer" containerID="d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24"
Sep 16 13:59:56.200002 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.199983    2580 scope.go:117] "RemoveContainer" containerID="d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24"
Sep 16 13:59:56.200270 ip-10-0-130-207 kubenswrapper[2580]: E0916 13:59:56.200252    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24\": container with ID starting with d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24 not found: ID does not exist" containerID="d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24"
Sep 16 13:59:56.200311 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.200280    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24"} err="failed to get container status \"d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24\": rpc error: code = NotFound desc = could not find container \"d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24\": container with ID starting with d24b0e25db397c81b679dbdd75e8943d94b39a952f48c41befd532f292ae3b24 not found: ID does not exist"
Sep 16 13:59:56.210254 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.210230    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"]
Sep 16 13:59:56.213038 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.213016    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2284bd/auth-test-model-2284bd-kserve-6c8c5bd47f-sjfx5"]
Sep 16 13:59:56.803514 ip-10-0-130-207 kubenswrapper[2580]: I0916 13:59:56.803480    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9f046f0-d92b-49c3-bfe0-7bb86208079b" path="/var/lib/kubelet/pods/a9f046f0-d92b-49c3-bfe0-7bb86208079b/volumes"
Sep 16 14:00:00.133035 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.132998    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"]
Sep 16 14:00:00.133420 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133366    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef73f2a4-84ec-4b34-acca-133473411a1a" containerName="payload-processing"
Sep 16 14:00:00.133420 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133380    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef73f2a4-84ec-4b34-acca-133473411a1a" containerName="payload-processing"
Sep 16 14:00:00.133420 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133388    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9f046f0-d92b-49c3-bfe0-7bb86208079b" containerName="main"
Sep 16 14:00:00.133420 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133394    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f046f0-d92b-49c3-bfe0-7bb86208079b" containerName="main"
Sep 16 14:00:00.133420 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133405    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea97e996-a2fe-4c6b-81c1-766645456f5c" containerName="payload-pre-processing"
Sep 16 14:00:00.133420 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133412    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea97e996-a2fe-4c6b-81c1-766645456f5c" containerName="payload-pre-processing"
Sep 16 14:00:00.133618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133428    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045e2ee0-b9d2-42df-a829-2a9727fefdb1" containerName="revoke-keys"
Sep 16 14:00:00.133618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133433    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="045e2ee0-b9d2-42df-a829-2a9727fefdb1" containerName="revoke-keys"
Sep 16 14:00:00.133618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133479    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="045e2ee0-b9d2-42df-a829-2a9727fefdb1" containerName="revoke-keys"
Sep 16 14:00:00.133618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133487    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef73f2a4-84ec-4b34-acca-133473411a1a" containerName="payload-processing"
Sep 16 14:00:00.133618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133494    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9f046f0-d92b-49c3-bfe0-7bb86208079b" containerName="main"
Sep 16 14:00:00.133618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.133500    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea97e996-a2fe-4c6b-81c1-766645456f5c" containerName="payload-pre-processing"
Sep 16 14:00:00.137466 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.137449    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"]
Sep 16 14:00:00.137567 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.137550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"
Sep 16 14:00:00.139689 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.139669    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 14:00:00.140411 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.140393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"
Sep 16 14:00:00.144055 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.144032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"]
Sep 16 14:00:00.145971 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.145949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"]
Sep 16 14:00:00.150276 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.150255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"
Sep 16 14:00:00.156762 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.156745    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"
Sep 16 14:00:00.421816 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.421642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"]
Sep 16 14:00:00.424576 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:00:00.424548    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c4718e_6273_49a6_880c_ecdf453134b8.slice/crio-2daf8d2f84741fb1d5c8a302b44502cfae694bd33e75a3d7ececa8daac50f169 WatchSource:0}: Error finding container 2daf8d2f84741fb1d5c8a302b44502cfae694bd33e75a3d7ececa8daac50f169: Status 404 returned error can't find the container with id 2daf8d2f84741fb1d5c8a302b44502cfae694bd33e75a3d7ececa8daac50f169
Sep 16 14:00:00.427093 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:00.427069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"]
Sep 16 14:00:00.429867 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:00:00.429841    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode183b128_d92b_4f2d_bf6b_ac0e8e2d2fd0.slice/crio-26e7b9c311568d505a0fa26c0769de26eb9826f3340bcf225d964d685435f761 WatchSource:0}: Error finding container 26e7b9c311568d505a0fa26c0769de26eb9826f3340bcf225d964d685435f761: Status 404 returned error can't find the container with id 26e7b9c311568d505a0fa26c0769de26eb9826f3340bcf225d964d685435f761
Sep 16 14:00:01.217183 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:01.217150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh" event={"ID":"e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0","Type":"ContainerStarted","Data":"0451efe1bef909c640edcd1af9943f83efc540d0c9916a162808c9ac69576695"}
Sep 16 14:00:01.217519 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:01.217213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh" event={"ID":"e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0","Type":"ContainerStarted","Data":"26e7b9c311568d505a0fa26c0769de26eb9826f3340bcf225d964d685435f761"}
Sep 16 14:00:01.218713 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:01.218686    2580 generic.go:356] "Generic (PLEG): container finished" podID="49c4718e-6273-49a6-880c-ecdf453134b8" containerID="0595d56b1816445adfe775d950cd7924cb027a5eaa267645c2d7d34b72a84115" exitCode=0
Sep 16 14:00:01.218798 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:01.218777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls" event={"ID":"49c4718e-6273-49a6-880c-ecdf453134b8","Type":"ContainerDied","Data":"0595d56b1816445adfe775d950cd7924cb027a5eaa267645c2d7d34b72a84115"}
Sep 16 14:00:01.218872 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:01.218832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls" event={"ID":"49c4718e-6273-49a6-880c-ecdf453134b8","Type":"ContainerStarted","Data":"2daf8d2f84741fb1d5c8a302b44502cfae694bd33e75a3d7ececa8daac50f169"}
Sep 16 14:00:01.231692 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:01.231645    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh" podStartSLOduration=1.231629562 podStartE2EDuration="1.231629562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:00:01.229957979 +0000 UTC m=+1628.913616373" watchObservedRunningTime="2026-09-16 14:00:01.231629562 +0000 UTC m=+1628.915287976"
Sep 16 14:00:02.225120 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:02.225083    2580 generic.go:356] "Generic (PLEG): container finished" podID="e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0" containerID="0451efe1bef909c640edcd1af9943f83efc540d0c9916a162808c9ac69576695" exitCode=0
Sep 16 14:00:02.225495 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:02.225141    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh" event={"ID":"e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0","Type":"ContainerDied","Data":"0451efe1bef909c640edcd1af9943f83efc540d0c9916a162808c9ac69576695"}
Sep 16 14:00:02.368530 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:02.368506    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"
Sep 16 14:00:03.230640 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:03.230611    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"
Sep 16 14:00:03.231055 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:03.230661    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls" event={"ID":"49c4718e-6273-49a6-880c-ecdf453134b8","Type":"ContainerDied","Data":"2daf8d2f84741fb1d5c8a302b44502cfae694bd33e75a3d7ececa8daac50f169"}
Sep 16 14:00:03.231055 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:03.230719    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2daf8d2f84741fb1d5c8a302b44502cfae694bd33e75a3d7ececa8daac50f169"
Sep 16 14:00:03.363045 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:03.363014    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"
Sep 16 14:00:04.237810 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:04.237777    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"
Sep 16 14:00:04.238250 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:04.237807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh" event={"ID":"e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0","Type":"ContainerDied","Data":"26e7b9c311568d505a0fa26c0769de26eb9826f3340bcf225d964d685435f761"}
Sep 16 14:00:04.238250 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:04.237860    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26e7b9c311568d505a0fa26c0769de26eb9826f3340bcf225d964d685435f761"
Sep 16 14:00:53.069191 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:53.069164    2580 scope.go:117] "RemoveContainer" containerID="9ae5528a66be80de00e80410c65687639ec757c8a576d4e571b38809fc1110da"
Sep 16 14:00:54.082947 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.082914    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7695-a2b27c"]
Sep 16 14:00:54.083444 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.083423    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0" containerName="cleanup"
Sep 16 14:00:54.083523 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.083447    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0" containerName="cleanup"
Sep 16 14:00:54.083523 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.083459    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c4718e-6273-49a6-880c-ecdf453134b8" containerName="cleanup"
Sep 16 14:00:54.083523 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.083467    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c4718e-6273-49a6-880c-ecdf453134b8" containerName="cleanup"
Sep 16 14:00:54.083679 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.083552    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c4718e-6273-49a6-880c-ecdf453134b8" containerName="cleanup"
Sep 16 14:00:54.083679 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.083564    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0" containerName="cleanup"
Sep 16 14:00:54.087738 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.087715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:00:54.089623 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.089603    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmn8d\""
Sep 16 14:00:54.092319 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.092295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7695-a2b27c"]
Sep 16 14:00:54.170617 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.170587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfgfk\" (UniqueName: \"kubernetes.io/projected/6d723cf3-054c-401b-81c4-ce779291e264-kube-api-access-jfgfk\") pod \"test-curl-7695-a2b27c\" (UID: \"6d723cf3-054c-401b-81c4-ce779291e264\") " pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:00:54.271242 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.271207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfgfk\" (UniqueName: \"kubernetes.io/projected/6d723cf3-054c-401b-81c4-ce779291e264-kube-api-access-jfgfk\") pod \"test-curl-7695-a2b27c\" (UID: \"6d723cf3-054c-401b-81c4-ce779291e264\") " pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:00:54.279530 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.279500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfgfk\" (UniqueName: \"kubernetes.io/projected/6d723cf3-054c-401b-81c4-ce779291e264-kube-api-access-jfgfk\") pod \"test-curl-7695-a2b27c\" (UID: \"6d723cf3-054c-401b-81c4-ce779291e264\") " pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:00:54.397682 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.397652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:00:54.664705 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:54.664631    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7695-a2b27c"]
Sep 16 14:00:54.668092 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:00:54.668063    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d723cf3_054c_401b_81c4_ce779291e264.slice/crio-82f4147334d9069fc7707485ef94cc4ff8c5c37244ffe69018c3646be373f2f2 WatchSource:0}: Error finding container 82f4147334d9069fc7707485ef94cc4ff8c5c37244ffe69018c3646be373f2f2: Status 404 returned error can't find the container with id 82f4147334d9069fc7707485ef94cc4ff8c5c37244ffe69018c3646be373f2f2
Sep 16 14:00:55.487764 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:55.487729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7695-a2b27c" event={"ID":"6d723cf3-054c-401b-81c4-ce779291e264","Type":"ContainerStarted","Data":"74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b"}
Sep 16 14:00:55.487764 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:55.487774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7695-a2b27c" event={"ID":"6d723cf3-054c-401b-81c4-ce779291e264","Type":"ContainerStarted","Data":"82f4147334d9069fc7707485ef94cc4ff8c5c37244ffe69018c3646be373f2f2"}
Sep 16 14:00:55.501932 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:55.501866    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7695-a2b27c" podStartSLOduration=1.5018512670000002 podStartE2EDuration="1.501851267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:00:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:00:55.500249336 +0000 UTC m=+1683.183907729" watchObservedRunningTime="2026-09-16 14:00:55.501851267 +0000 UTC m=+1683.185509660"
Sep 16 14:00:56.080945 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:56.080909    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7695-a2b27c"]
Sep 16 14:00:56.083485 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:56.083463    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7695-a2b27c"]
Sep 16 14:00:56.492995 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:00:56.492961    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7695-a2b27c" container="test-curl-7695-a2b27c"
Sep 16 14:00:57.497034 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:57.496991    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7695-a2b27c" podUID="6d723cf3-054c-401b-81c4-ce779291e264" containerName="test-curl-7695-a2b27c" containerID="cri-o://74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b" gracePeriod=2
Sep 16 14:00:59.655974 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:59.655946    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:00:59.715084 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:59.715047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfgfk\" (UniqueName: \"kubernetes.io/projected/6d723cf3-054c-401b-81c4-ce779291e264-kube-api-access-jfgfk\") pod \"6d723cf3-054c-401b-81c4-ce779291e264\" (UID: \"6d723cf3-054c-401b-81c4-ce779291e264\") "
Sep 16 14:00:59.717118 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:59.717093    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d723cf3-054c-401b-81c4-ce779291e264-kube-api-access-jfgfk" (OuterVolumeSpecName: "kube-api-access-jfgfk") pod "6d723cf3-054c-401b-81c4-ce779291e264" (UID: "6d723cf3-054c-401b-81c4-ce779291e264"). InnerVolumeSpecName "kube-api-access-jfgfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:00:59.816646 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:00:59.816564    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfgfk\" (UniqueName: \"kubernetes.io/projected/6d723cf3-054c-401b-81c4-ce779291e264-kube-api-access-jfgfk\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:01:00.518817 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:01:00.518786    2580 generic.go:356] "Generic (PLEG): container finished" podID="6d723cf3-054c-401b-81c4-ce779291e264" containerID="74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b" exitCode=137
Sep 16 14:01:00.519025 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:01:00.518829    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:01:00.519025 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:01:00.518894    2580 scope.go:117] "RemoveContainer" containerID="74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b"
Sep 16 14:01:00.528483 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:01:00.528463    2580 scope.go:117] "RemoveContainer" containerID="74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b"
Sep 16 14:01:00.528808 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:01:00.528785    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b\": container with ID starting with 74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b not found: ID does not exist" containerID="74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b"
Sep 16 14:01:00.528862 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:01:00.528820    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b"} err="failed to get container status \"74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b\": rpc error: code = NotFound desc = could not find container \"74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b\": container with ID starting with 74a097df903ab03e9c57d1a8c7152d5c83e18872c9bb0005233744ffe99c751b not found: ID does not exist"
Sep 16 14:01:00.529360 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:01:00.529338    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7695-a2b27c\" is forbidden: User \"system:node:ip-10-0-130-207.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-207.ec2.internal' and this object" podUID="6d723cf3-054c-401b-81c4-ce779291e264" pod="opendatahub/test-curl-7695-a2b27c"
Sep 16 14:01:00.804328 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:01:00.804248    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d723cf3-054c-401b-81c4-ce779291e264" path="/var/lib/kubelet/pods/6d723cf3-054c-401b-81c4-ce779291e264/volumes"
Sep 16 14:02:16.950276 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.950241    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"]
Sep 16 14:02:16.950719 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.950561    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d723cf3-054c-401b-81c4-ce779291e264" containerName="test-curl-7695-a2b27c"
Sep 16 14:02:16.950719 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.950572    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d723cf3-054c-401b-81c4-ce779291e264" containerName="test-curl-7695-a2b27c"
Sep 16 14:02:16.950719 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.950632    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d723cf3-054c-401b-81c4-ce779291e264" containerName="test-curl-7695-a2b27c"
Sep 16 14:02:16.953502 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.953484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:16.955661 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.955637    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-2ed470\""
Sep 16 14:02:16.955777 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.955696    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-2ed470-dockercfg-vbx5j\""
Sep 16 14:02:16.961641 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.961613    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"]
Sep 16 14:02:16.983942 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.983916    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"]
Sep 16 14:02:16.986981 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.986966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:16.997613 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:16.997592    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"]
Sep 16 14:02:17.055788 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.055751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvj4\" (UniqueName: \"kubernetes.io/projected/088a34be-9f50-4d5b-aa0a-66bc790d18c4-kube-api-access-9nvj4\") pod \"payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.055998 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.055802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/088a34be-9f50-4d5b-aa0a-66bc790d18c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.157284 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.157244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.157461 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.157293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28whp\" (UniqueName: \"kubernetes.io/projected/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-kube-api-access-28whp\") pod \"payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.157461 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.157318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvj4\" (UniqueName: \"kubernetes.io/projected/088a34be-9f50-4d5b-aa0a-66bc790d18c4-kube-api-access-9nvj4\") pod \"payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.157461 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.157383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/088a34be-9f50-4d5b-aa0a-66bc790d18c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.158049 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.158019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/088a34be-9f50-4d5b-aa0a-66bc790d18c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.166749 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.166723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvj4\" (UniqueName: \"kubernetes.io/projected/088a34be-9f50-4d5b-aa0a-66bc790d18c4-kube-api-access-9nvj4\") pod \"payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.258571 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.258468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.258571 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.258527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28whp\" (UniqueName: \"kubernetes.io/projected/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-kube-api-access-28whp\") pod \"payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.259369 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.259341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.265104 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.265071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.265392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.265372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28whp\" (UniqueName: \"kubernetes.io/projected/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-kube-api-access-28whp\") pod \"payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.296204 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.296174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.539492 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.539464    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"]
Sep 16 14:02:17.541999 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:02:17.541969    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088a34be_9f50_4d5b_aa0a_66bc790d18c4.slice/crio-51c9e69b106afc8a7bf3bc053ec4e379c90c6678b9f0c6e1cb00e59446803d3f WatchSource:0}: Error finding container 51c9e69b106afc8a7bf3bc053ec4e379c90c6678b9f0c6e1cb00e59446803d3f: Status 404 returned error can't find the container with id 51c9e69b106afc8a7bf3bc053ec4e379c90c6678b9f0c6e1cb00e59446803d3f
Sep 16 14:02:17.579656 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.579481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"]
Sep 16 14:02:17.594001 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:02:17.593972    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11498b4_a14c_4c68_bd87_750d9d7d5a0e.slice/crio-facdb438acebb64681eaba97913f7fa281f0708cacee1403a00d87b0aec3838b WatchSource:0}: Error finding container facdb438acebb64681eaba97913f7fa281f0708cacee1403a00d87b0aec3838b: Status 404 returned error can't find the container with id facdb438acebb64681eaba97913f7fa281f0708cacee1403a00d87b0aec3838b
Sep 16 14:02:17.886054 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.886017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq" event={"ID":"088a34be-9f50-4d5b-aa0a-66bc790d18c4","Type":"ContainerStarted","Data":"944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9"}
Sep 16 14:02:17.886238 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.886079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq" event={"ID":"088a34be-9f50-4d5b-aa0a-66bc790d18c4","Type":"ContainerStarted","Data":"51c9e69b106afc8a7bf3bc053ec4e379c90c6678b9f0c6e1cb00e59446803d3f"}
Sep 16 14:02:17.886238 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.886110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:17.887711 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.887685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l" event={"ID":"a11498b4-a14c-4c68-bd87-750d9d7d5a0e","Type":"ContainerStarted","Data":"8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60"}
Sep 16 14:02:17.887831 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.887732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l" event={"ID":"a11498b4-a14c-4c68-bd87-750d9d7d5a0e","Type":"ContainerStarted","Data":"facdb438acebb64681eaba97913f7fa281f0708cacee1403a00d87b0aec3838b"}
Sep 16 14:02:17.887831 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.887809    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:17.900778 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.900725    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq" podStartSLOduration=1.900710914 podStartE2EDuration="1.900710914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:02:17.899537958 +0000 UTC m=+1765.583196353" watchObservedRunningTime="2026-09-16 14:02:17.900710914 +0000 UTC m=+1765.584369371"
Sep 16 14:02:17.916600 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:17.916550    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l" podStartSLOduration=1.916532799 podStartE2EDuration="1.916532799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:02:17.916033245 +0000 UTC m=+1765.599691639" watchObservedRunningTime="2026-09-16 14:02:17.916532799 +0000 UTC m=+1765.600191197"
Sep 16 14:02:28.893684 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:28.893652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:02:28.894185 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:28.893807    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:02:47.838324 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:47.838242    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"]
Sep 16 14:02:47.840723 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:47.840702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:47.842644 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:47.842623    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-9e8244\""
Sep 16 14:02:47.843058 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:47.843041    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-9e8244-dockercfg-bpgct\""
Sep 16 14:02:47.852357 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:47.852331    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"]
Sep 16 14:02:47.922347 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:47.922305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98bc8ddb-9e5c-4be1-b48c-53168adda04f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:47.922523 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:47.922356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxl9m\" (UniqueName: \"kubernetes.io/projected/98bc8ddb-9e5c-4be1-b48c-53168adda04f-kube-api-access-wxl9m\") pod \"payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:48.023736 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:48.023686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98bc8ddb-9e5c-4be1-b48c-53168adda04f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:48.023931 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:48.023762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxl9m\" (UniqueName: \"kubernetes.io/projected/98bc8ddb-9e5c-4be1-b48c-53168adda04f-kube-api-access-wxl9m\") pod \"payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:48.024414 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:48.024393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98bc8ddb-9e5c-4be1-b48c-53168adda04f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:48.031195 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:48.031163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxl9m\" (UniqueName: \"kubernetes.io/projected/98bc8ddb-9e5c-4be1-b48c-53168adda04f-kube-api-access-wxl9m\") pod \"payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:48.151339 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:48.151302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:48.421248 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:48.421223    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"]
Sep 16 14:02:48.423849 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:02:48.423821    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bc8ddb_9e5c_4be1_b48c_53168adda04f.slice/crio-5a49cd3f626b0535f7edba03c5a7c4311f264e75601cc49bb77010c05499e831 WatchSource:0}: Error finding container 5a49cd3f626b0535f7edba03c5a7c4311f264e75601cc49bb77010c05499e831: Status 404 returned error can't find the container with id 5a49cd3f626b0535f7edba03c5a7c4311f264e75601cc49bb77010c05499e831
Sep 16 14:02:49.041824 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:49.041790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5" event={"ID":"98bc8ddb-9e5c-4be1-b48c-53168adda04f","Type":"ContainerStarted","Data":"3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203"}
Sep 16 14:02:49.042200 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:49.041847    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5" event={"ID":"98bc8ddb-9e5c-4be1-b48c-53168adda04f","Type":"ContainerStarted","Data":"5a49cd3f626b0535f7edba03c5a7c4311f264e75601cc49bb77010c05499e831"}
Sep 16 14:02:49.042200 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:49.041965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:02:49.057272 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:49.057229    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5" podStartSLOduration=2.057215384 podStartE2EDuration="2.057215384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:02:49.056197962 +0000 UTC m=+1796.739856355" watchObservedRunningTime="2026-09-16 14:02:49.057215384 +0000 UTC m=+1796.740873777"
Sep 16 14:02:52.819026 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:52.818986    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 14:02:52.819876 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:02:52.819845    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 14:03:00.048365 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:00.048334    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:03:47.234788 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:47.234756    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l_a11498b4-a14c-4c68-bd87-750d9d7d5a0e/payload-processing/0.log"
Sep 16 14:03:53.109124 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:53.109091    2580 scope.go:117] "RemoveContainer" containerID="c1491827df124ef9e2eec84930172af5067e1e1aeb654276a778c848ed5d334f"
Sep 16 14:03:55.044560 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.044527    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"]
Sep 16 14:03:55.049222 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.049201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.051109 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.051084    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-11bfbd\""
Sep 16 14:03:55.051214 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.051084    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-11bfbd-dockercfg-4m9sf\""
Sep 16 14:03:55.057347 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.057322    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"]
Sep 16 14:03:55.075487 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.075459    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"]
Sep 16 14:03:55.078956 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.078940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.089310 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.089288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"]
Sep 16 14:03:55.159811 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.159783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xl26\" (UniqueName: \"kubernetes.io/projected/469db126-d0b6-46fa-b535-4fe4989b2e13-kube-api-access-5xl26\") pod \"payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.159950 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.159822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b88d5fa3-3c08-44f5-aa5f-dba022b77324-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.159950 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.159860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469db126-d0b6-46fa-b535-4fe4989b2e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.159950 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.159891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckf9r\" (UniqueName: \"kubernetes.io/projected/b88d5fa3-3c08-44f5-aa5f-dba022b77324-kube-api-access-ckf9r\") pod \"payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.260373 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.260343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckf9r\" (UniqueName: \"kubernetes.io/projected/b88d5fa3-3c08-44f5-aa5f-dba022b77324-kube-api-access-ckf9r\") pod \"payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.260531 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.260394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xl26\" (UniqueName: \"kubernetes.io/projected/469db126-d0b6-46fa-b535-4fe4989b2e13-kube-api-access-5xl26\") pod \"payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.260633 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.260616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b88d5fa3-3c08-44f5-aa5f-dba022b77324-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.260671 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.260656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469db126-d0b6-46fa-b535-4fe4989b2e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.261329 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.261310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469db126-d0b6-46fa-b535-4fe4989b2e13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.261413 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.261332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b88d5fa3-3c08-44f5-aa5f-dba022b77324-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.267893 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.267863    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckf9r\" (UniqueName: \"kubernetes.io/projected/b88d5fa3-3c08-44f5-aa5f-dba022b77324-kube-api-access-ckf9r\") pod \"payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.268023 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.267988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xl26\" (UniqueName: \"kubernetes.io/projected/469db126-d0b6-46fa-b535-4fe4989b2e13-kube-api-access-5xl26\") pod \"payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.361127 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.361064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:55.388815 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.388784    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:55.633683 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.633659    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"]
Sep 16 14:03:55.636318 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:03:55.636293    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469db126_d0b6_46fa_b535_4fe4989b2e13.slice/crio-273c32e164f56821b4bba4c6206914ed8cf86ee122fde591bf34adda1ea5fb69 WatchSource:0}: Error finding container 273c32e164f56821b4bba4c6206914ed8cf86ee122fde591bf34adda1ea5fb69: Status 404 returned error can't find the container with id 273c32e164f56821b4bba4c6206914ed8cf86ee122fde591bf34adda1ea5fb69
Sep 16 14:03:55.667271 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:55.667244    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"]
Sep 16 14:03:55.669451 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:03:55.669427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb88d5fa3_3c08_44f5_aa5f_dba022b77324.slice/crio-8d829a57e59e454fade5d893b9fb265cff9e4b4b67421c648bb6c5f6d52bfba4 WatchSource:0}: Error finding container 8d829a57e59e454fade5d893b9fb265cff9e4b4b67421c648bb6c5f6d52bfba4: Status 404 returned error can't find the container with id 8d829a57e59e454fade5d893b9fb265cff9e4b4b67421c648bb6c5f6d52bfba4
Sep 16 14:03:56.373070 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.373024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh" event={"ID":"469db126-d0b6-46fa-b535-4fe4989b2e13","Type":"ContainerStarted","Data":"643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70"}
Sep 16 14:03:56.373476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.373119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh" event={"ID":"469db126-d0b6-46fa-b535-4fe4989b2e13","Type":"ContainerStarted","Data":"273c32e164f56821b4bba4c6206914ed8cf86ee122fde591bf34adda1ea5fb69"}
Sep 16 14:03:56.373476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.373149    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:03:56.374523 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.374497    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld" event={"ID":"b88d5fa3-3c08-44f5-aa5f-dba022b77324","Type":"ContainerStarted","Data":"bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5"}
Sep 16 14:03:56.374635 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.374539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld" event={"ID":"b88d5fa3-3c08-44f5-aa5f-dba022b77324","Type":"ContainerStarted","Data":"8d829a57e59e454fade5d893b9fb265cff9e4b4b67421c648bb6c5f6d52bfba4"}
Sep 16 14:03:56.374635 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.374581    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:03:56.388261 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.388212    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh" podStartSLOduration=1.38819559 podStartE2EDuration="1.38819559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:03:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:03:56.387104717 +0000 UTC m=+1864.070763113" watchObservedRunningTime="2026-09-16 14:03:56.38819559 +0000 UTC m=+1864.071853986"
Sep 16 14:03:56.401679 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:03:56.401638    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld" podStartSLOduration=1.401625341 podStartE2EDuration="1.401625341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:03:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:03:56.400959611 +0000 UTC m=+1864.084618007" watchObservedRunningTime="2026-09-16 14:03:56.401625341 +0000 UTC m=+1864.085283750"
Sep 16 14:04:07.381642 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:07.381609    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:04:07.382103 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:07.381766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:04:17.144820 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.144739    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"]
Sep 16 14:04:17.148079 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.148062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"
Sep 16 14:04:17.150183 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.150162    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:04:17.150281 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.150178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 14:04:17.154754 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.154732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"]
Sep 16 14:04:17.243769 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.243734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/857ff3d0-409e-4712-a941-aeae61acdc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv\" (UID: \"857ff3d0-409e-4712-a941-aeae61acdc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"
Sep 16 14:04:17.344499 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.344458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/857ff3d0-409e-4712-a941-aeae61acdc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv\" (UID: \"857ff3d0-409e-4712-a941-aeae61acdc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"
Sep 16 14:04:17.345163 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.345140    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/857ff3d0-409e-4712-a941-aeae61acdc40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv\" (UID: \"857ff3d0-409e-4712-a941-aeae61acdc40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"
Sep 16 14:04:17.459867 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.459787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"
Sep 16 14:04:17.733093 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:17.733069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"]
Sep 16 14:04:17.735713 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:04:17.735685    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857ff3d0_409e_4712_a941_aeae61acdc40.slice/crio-b38a07d30aa7834d994b69187e0059569f03f0eadc3be6541f47f43fbff84aa1 WatchSource:0}: Error finding container b38a07d30aa7834d994b69187e0059569f03f0eadc3be6541f47f43fbff84aa1: Status 404 returned error can't find the container with id b38a07d30aa7834d994b69187e0059569f03f0eadc3be6541f47f43fbff84aa1
Sep 16 14:04:18.485771 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:18.485733    2580 generic.go:356] "Generic (PLEG): container finished" podID="857ff3d0-409e-4712-a941-aeae61acdc40" containerID="532668f4ab68394cc548dbce885dd3302c6adfebf9090cbf411195f296d2d4b7" exitCode=0
Sep 16 14:04:18.486148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:18.485769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv" event={"ID":"857ff3d0-409e-4712-a941-aeae61acdc40","Type":"ContainerDied","Data":"532668f4ab68394cc548dbce885dd3302c6adfebf9090cbf411195f296d2d4b7"}
Sep 16 14:04:18.486148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:18.485832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv" event={"ID":"857ff3d0-409e-4712-a941-aeae61acdc40","Type":"ContainerStarted","Data":"b38a07d30aa7834d994b69187e0059569f03f0eadc3be6541f47f43fbff84aa1"}
Sep 16 14:04:19.628980 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:19.628956    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"
Sep 16 14:04:19.767410 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:19.767330    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/857ff3d0-409e-4712-a941-aeae61acdc40-maas-api-service-ca\") pod \"857ff3d0-409e-4712-a941-aeae61acdc40\" (UID: \"857ff3d0-409e-4712-a941-aeae61acdc40\") "
Sep 16 14:04:19.767776 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:19.767749    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/857ff3d0-409e-4712-a941-aeae61acdc40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "857ff3d0-409e-4712-a941-aeae61acdc40" (UID: "857ff3d0-409e-4712-a941-aeae61acdc40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:04:19.868317 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:19.868287    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/857ff3d0-409e-4712-a941-aeae61acdc40-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:20.497026 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:20.496991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv" event={"ID":"857ff3d0-409e-4712-a941-aeae61acdc40","Type":"ContainerDied","Data":"b38a07d30aa7834d994b69187e0059569f03f0eadc3be6541f47f43fbff84aa1"}
Sep 16 14:04:20.497026 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:20.497016    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"
Sep 16 14:04:20.497227 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:20.497071    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b38a07d30aa7834d994b69187e0059569f03f0eadc3be6541f47f43fbff84aa1"
Sep 16 14:04:23.391537 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.391500    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"]
Sep 16 14:04:23.391958 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.391777    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld" podUID="b88d5fa3-3c08-44f5-aa5f-dba022b77324" containerName="payload-processing" containerID="cri-o://bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5" gracePeriod=30
Sep 16 14:04:23.398351 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.398325    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"]
Sep 16 14:04:23.398829 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.398543    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh" podUID="469db126-d0b6-46fa-b535-4fe4989b2e13" containerName="payload-pre-processing" containerID="cri-o://643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70" gracePeriod=30
Sep 16 14:04:23.562236 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.562199    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"]
Sep 16 14:04:23.564920 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.564879    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-11bfbd-7xsxv"]
Sep 16 14:04:23.678889 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.678859    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:04:23.682482 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.682464    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:04:23.803956 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.803920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xl26\" (UniqueName: \"kubernetes.io/projected/469db126-d0b6-46fa-b535-4fe4989b2e13-kube-api-access-5xl26\") pod \"469db126-d0b6-46fa-b535-4fe4989b2e13\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") "
Sep 16 14:04:23.804123 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.803962    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckf9r\" (UniqueName: \"kubernetes.io/projected/b88d5fa3-3c08-44f5-aa5f-dba022b77324-kube-api-access-ckf9r\") pod \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") "
Sep 16 14:04:23.804123 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.804017    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b88d5fa3-3c08-44f5-aa5f-dba022b77324-plugins-config-volume\") pod \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\" (UID: \"b88d5fa3-3c08-44f5-aa5f-dba022b77324\") "
Sep 16 14:04:23.804123 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.804074    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469db126-d0b6-46fa-b535-4fe4989b2e13-plugins-config-volume\") pod \"469db126-d0b6-46fa-b535-4fe4989b2e13\" (UID: \"469db126-d0b6-46fa-b535-4fe4989b2e13\") "
Sep 16 14:04:23.804467 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.804435    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b88d5fa3-3c08-44f5-aa5f-dba022b77324-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b88d5fa3-3c08-44f5-aa5f-dba022b77324" (UID: "b88d5fa3-3c08-44f5-aa5f-dba022b77324"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:04:23.804600 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.804499    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/469db126-d0b6-46fa-b535-4fe4989b2e13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "469db126-d0b6-46fa-b535-4fe4989b2e13" (UID: "469db126-d0b6-46fa-b535-4fe4989b2e13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:04:23.806176 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.806152    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469db126-d0b6-46fa-b535-4fe4989b2e13-kube-api-access-5xl26" (OuterVolumeSpecName: "kube-api-access-5xl26") pod "469db126-d0b6-46fa-b535-4fe4989b2e13" (UID: "469db126-d0b6-46fa-b535-4fe4989b2e13"). InnerVolumeSpecName "kube-api-access-5xl26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:04:23.806294 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.806188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b88d5fa3-3c08-44f5-aa5f-dba022b77324-kube-api-access-ckf9r" (OuterVolumeSpecName: "kube-api-access-ckf9r") pod "b88d5fa3-3c08-44f5-aa5f-dba022b77324" (UID: "b88d5fa3-3c08-44f5-aa5f-dba022b77324"). InnerVolumeSpecName "kube-api-access-ckf9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:04:23.905148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.905073    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xl26\" (UniqueName: \"kubernetes.io/projected/469db126-d0b6-46fa-b535-4fe4989b2e13-kube-api-access-5xl26\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:23.905148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.905100    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckf9r\" (UniqueName: \"kubernetes.io/projected/b88d5fa3-3c08-44f5-aa5f-dba022b77324-kube-api-access-ckf9r\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:23.905148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.905109    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b88d5fa3-3c08-44f5-aa5f-dba022b77324-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:23.905148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:23.905118    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/469db126-d0b6-46fa-b535-4fe4989b2e13-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:24.518485 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.518453    2580 generic.go:356] "Generic (PLEG): container finished" podID="b88d5fa3-3c08-44f5-aa5f-dba022b77324" containerID="bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5" exitCode=0
Sep 16 14:04:24.518982 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.518497    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"
Sep 16 14:04:24.518982 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.518542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld" event={"ID":"b88d5fa3-3c08-44f5-aa5f-dba022b77324","Type":"ContainerDied","Data":"bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5"}
Sep 16 14:04:24.518982 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.518607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld" event={"ID":"b88d5fa3-3c08-44f5-aa5f-dba022b77324","Type":"ContainerDied","Data":"8d829a57e59e454fade5d893b9fb265cff9e4b4b67421c648bb6c5f6d52bfba4"}
Sep 16 14:04:24.518982 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.518645    2580 scope.go:117] "RemoveContainer" containerID="bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5"
Sep 16 14:04:24.520134 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.520035    2580 generic.go:356] "Generic (PLEG): container finished" podID="469db126-d0b6-46fa-b535-4fe4989b2e13" containerID="643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70" exitCode=0
Sep 16 14:04:24.520134 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.520058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh" event={"ID":"469db126-d0b6-46fa-b535-4fe4989b2e13","Type":"ContainerDied","Data":"643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70"}
Sep 16 14:04:24.520134 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.520080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"
Sep 16 14:04:24.520134 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.520111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh" event={"ID":"469db126-d0b6-46fa-b535-4fe4989b2e13","Type":"ContainerDied","Data":"273c32e164f56821b4bba4c6206914ed8cf86ee122fde591bf34adda1ea5fb69"}
Sep 16 14:04:24.528701 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.528668    2580 scope.go:117] "RemoveContainer" containerID="bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5"
Sep 16 14:04:24.529090 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:04:24.529050    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5\": container with ID starting with bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5 not found: ID does not exist" containerID="bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5"
Sep 16 14:04:24.529148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.529104    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5"} err="failed to get container status \"bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5\": rpc error: code = NotFound desc = could not find container \"bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5\": container with ID starting with bcec9d76de52ec556857cc7745f6cc4cfcd517ba8b31523839ad8627fbb4cee5 not found: ID does not exist"
Sep 16 14:04:24.529148 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.529128    2580 scope.go:117] "RemoveContainer" containerID="643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70"
Sep 16 14:04:24.537526 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.537510    2580 scope.go:117] "RemoveContainer" containerID="643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70"
Sep 16 14:04:24.537799 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:04:24.537780    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70\": container with ID starting with 643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70 not found: ID does not exist" containerID="643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70"
Sep 16 14:04:24.537865 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.537811    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70"} err="failed to get container status \"643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70\": rpc error: code = NotFound desc = could not find container \"643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70\": container with ID starting with 643b34f01280aa579f151a5e8aa7cef6191bf300becc1729bad2f2a3d99e9d70 not found: ID does not exist"
Sep 16 14:04:24.556841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.551660    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"]
Sep 16 14:04:24.556841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.556101    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-11bfbd-769648f446-g6pdh"]
Sep 16 14:04:24.562278 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.562250    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"]
Sep 16 14:04:24.565626 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.565600    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-11bfbd-c9bf66495-79nld"]
Sep 16 14:04:24.802545 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.802461    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469db126-d0b6-46fa-b535-4fe4989b2e13" path="/var/lib/kubelet/pods/469db126-d0b6-46fa-b535-4fe4989b2e13/volumes"
Sep 16 14:04:24.802831 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.802817    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="857ff3d0-409e-4712-a941-aeae61acdc40" path="/var/lib/kubelet/pods/857ff3d0-409e-4712-a941-aeae61acdc40/volumes"
Sep 16 14:04:24.803171 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:24.803159    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b88d5fa3-3c08-44f5-aa5f-dba022b77324" path="/var/lib/kubelet/pods/b88d5fa3-3c08-44f5-aa5f-dba022b77324/volumes"
Sep 16 14:04:34.178092 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178057    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"]
Sep 16 14:04:34.178476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178384    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469db126-d0b6-46fa-b535-4fe4989b2e13" containerName="payload-pre-processing"
Sep 16 14:04:34.178476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178395    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="469db126-d0b6-46fa-b535-4fe4989b2e13" containerName="payload-pre-processing"
Sep 16 14:04:34.178476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178403    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b88d5fa3-3c08-44f5-aa5f-dba022b77324" containerName="payload-processing"
Sep 16 14:04:34.178476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178409    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b88d5fa3-3c08-44f5-aa5f-dba022b77324" containerName="payload-processing"
Sep 16 14:04:34.178476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178422    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857ff3d0-409e-4712-a941-aeae61acdc40" containerName="revoke-keys"
Sep 16 14:04:34.178476 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178427    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="857ff3d0-409e-4712-a941-aeae61acdc40" containerName="revoke-keys"
Sep 16 14:04:34.178708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178493    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b88d5fa3-3c08-44f5-aa5f-dba022b77324" containerName="payload-processing"
Sep 16 14:04:34.178708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178501    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="857ff3d0-409e-4712-a941-aeae61acdc40" containerName="revoke-keys"
Sep 16 14:04:34.178708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.178508    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="469db126-d0b6-46fa-b535-4fe4989b2e13" containerName="payload-pre-processing"
Sep 16 14:04:34.181431 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.181414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"
Sep 16 14:04:34.183328 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.183306    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:04:34.183462 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.183340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 14:04:34.186126 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.186104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf\" (UID: \"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"
Sep 16 14:04:34.187008 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.186987    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"]
Sep 16 14:04:34.287149 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.287120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf\" (UID: \"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"
Sep 16 14:04:34.287691 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.287671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf\" (UID: \"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"
Sep 16 14:04:34.492251 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.492173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"
Sep 16 14:04:34.760335 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:34.760261    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"]
Sep 16 14:04:34.763422 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:04:34.763394    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69d86d1_dbf0_4de7_b076_f8f4acdfbbe8.slice/crio-ba5452d39a75b5c42b3b6cde44d39bdf463f0884175f8f7da8140c7156d95bf4 WatchSource:0}: Error finding container ba5452d39a75b5c42b3b6cde44d39bdf463f0884175f8f7da8140c7156d95bf4: Status 404 returned error can't find the container with id ba5452d39a75b5c42b3b6cde44d39bdf463f0884175f8f7da8140c7156d95bf4
Sep 16 14:04:35.575652 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:35.575622    2580 generic.go:356] "Generic (PLEG): container finished" podID="f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8" containerID="c4e110d7d8a7dd5f8ccb79a6907d1a84b43283480ed755c8f1435157f9450729" exitCode=0
Sep 16 14:04:35.575983 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:35.575682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf" event={"ID":"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8","Type":"ContainerDied","Data":"c4e110d7d8a7dd5f8ccb79a6907d1a84b43283480ed755c8f1435157f9450729"}
Sep 16 14:04:35.575983 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:35.575721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf" event={"ID":"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8","Type":"ContainerStarted","Data":"ba5452d39a75b5c42b3b6cde44d39bdf463f0884175f8f7da8140c7156d95bf4"}
Sep 16 14:04:36.704360 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:36.704333    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"
Sep 16 14:04:36.805639 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:36.805607    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8-maas-api-service-ca\") pod \"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8\" (UID: \"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8\") "
Sep 16 14:04:36.805934 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:36.805911    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8" (UID: "f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:04:36.906329 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:36.906299    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:37.586487 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:37.586454    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"
Sep 16 14:04:37.586677 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:37.586490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf" event={"ID":"f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8","Type":"ContainerDied","Data":"ba5452d39a75b5c42b3b6cde44d39bdf463f0884175f8f7da8140c7156d95bf4"}
Sep 16 14:04:37.586677 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:37.586538    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba5452d39a75b5c42b3b6cde44d39bdf463f0884175f8f7da8140c7156d95bf4"
Sep 16 14:04:49.722126 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:49.722090    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"]
Sep 16 14:04:49.722742 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:49.722349    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l" podUID="a11498b4-a14c-4c68-bd87-750d9d7d5a0e" containerName="payload-processing" containerID="cri-o://8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60" gracePeriod=30
Sep 16 14:04:49.730258 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:49.730101    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"]
Sep 16 14:04:49.730668 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:49.730601    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq" podUID="088a34be-9f50-4d5b-aa0a-66bc790d18c4" containerName="payload-pre-processing" containerID="cri-o://944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9" gracePeriod=30
Sep 16 14:04:49.894453 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:49.893532    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"]
Sep 16 14:04:49.896552 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:49.896364    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2ed470-fvdkf"]
Sep 16 14:04:49.979091 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:49.979027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:04:50.005914 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.005878    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:04:50.016005 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.015971    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-plugins-config-volume\") pod \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") "
Sep 16 14:04:50.016187 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.016070    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28whp\" (UniqueName: \"kubernetes.io/projected/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-kube-api-access-28whp\") pod \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\" (UID: \"a11498b4-a14c-4c68-bd87-750d9d7d5a0e\") "
Sep 16 14:04:50.016383 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.016356    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a11498b4-a14c-4c68-bd87-750d9d7d5a0e" (UID: "a11498b4-a14c-4c68-bd87-750d9d7d5a0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:04:50.018172 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.018152    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-kube-api-access-28whp" (OuterVolumeSpecName: "kube-api-access-28whp") pod "a11498b4-a14c-4c68-bd87-750d9d7d5a0e" (UID: "a11498b4-a14c-4c68-bd87-750d9d7d5a0e"). InnerVolumeSpecName "kube-api-access-28whp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:04:50.117015 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.116978    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nvj4\" (UniqueName: \"kubernetes.io/projected/088a34be-9f50-4d5b-aa0a-66bc790d18c4-kube-api-access-9nvj4\") pod \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") "
Sep 16 14:04:50.117015 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.117015    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/088a34be-9f50-4d5b-aa0a-66bc790d18c4-plugins-config-volume\") pod \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\" (UID: \"088a34be-9f50-4d5b-aa0a-66bc790d18c4\") "
Sep 16 14:04:50.117287 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.117258    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:50.117353 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.117293    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28whp\" (UniqueName: \"kubernetes.io/projected/a11498b4-a14c-4c68-bd87-750d9d7d5a0e-kube-api-access-28whp\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:50.117433 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.117410    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/088a34be-9f50-4d5b-aa0a-66bc790d18c4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "088a34be-9f50-4d5b-aa0a-66bc790d18c4" (UID: "088a34be-9f50-4d5b-aa0a-66bc790d18c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:04:50.119165 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.119142    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/088a34be-9f50-4d5b-aa0a-66bc790d18c4-kube-api-access-9nvj4" (OuterVolumeSpecName: "kube-api-access-9nvj4") pod "088a34be-9f50-4d5b-aa0a-66bc790d18c4" (UID: "088a34be-9f50-4d5b-aa0a-66bc790d18c4"). InnerVolumeSpecName "kube-api-access-9nvj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:04:50.218006 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.217972    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nvj4\" (UniqueName: \"kubernetes.io/projected/088a34be-9f50-4d5b-aa0a-66bc790d18c4-kube-api-access-9nvj4\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:50.218006 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.218001    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/088a34be-9f50-4d5b-aa0a-66bc790d18c4-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:04:50.661772 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.661736    2580 generic.go:356] "Generic (PLEG): container finished" podID="a11498b4-a14c-4c68-bd87-750d9d7d5a0e" containerID="8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60" exitCode=0
Sep 16 14:04:50.661993 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.661784    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"
Sep 16 14:04:50.661993 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.661828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l" event={"ID":"a11498b4-a14c-4c68-bd87-750d9d7d5a0e","Type":"ContainerDied","Data":"8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60"}
Sep 16 14:04:50.661993 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.661913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l" event={"ID":"a11498b4-a14c-4c68-bd87-750d9d7d5a0e","Type":"ContainerDied","Data":"facdb438acebb64681eaba97913f7fa281f0708cacee1403a00d87b0aec3838b"}
Sep 16 14:04:50.661993 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.661948    2580 scope.go:117] "RemoveContainer" containerID="8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60"
Sep 16 14:04:50.663324 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.663303    2580 generic.go:356] "Generic (PLEG): container finished" podID="088a34be-9f50-4d5b-aa0a-66bc790d18c4" containerID="944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9" exitCode=0
Sep 16 14:04:50.663407 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.663334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq" event={"ID":"088a34be-9f50-4d5b-aa0a-66bc790d18c4","Type":"ContainerDied","Data":"944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9"}
Sep 16 14:04:50.663407 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.663356    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"
Sep 16 14:04:50.663407 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.663378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq" event={"ID":"088a34be-9f50-4d5b-aa0a-66bc790d18c4","Type":"ContainerDied","Data":"51c9e69b106afc8a7bf3bc053ec4e379c90c6678b9f0c6e1cb00e59446803d3f"}
Sep 16 14:04:50.671255 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.671234    2580 scope.go:117] "RemoveContainer" containerID="8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60"
Sep 16 14:04:50.671663 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:04:50.671636    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60\": container with ID starting with 8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60 not found: ID does not exist" containerID="8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60"
Sep 16 14:04:50.671720 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.671670    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60"} err="failed to get container status \"8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60\": rpc error: code = NotFound desc = could not find container \"8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60\": container with ID starting with 8bb5952b31a78e6d3861b0b91c928a41a9ca4bee0f7c22c8906da8bd8d745b60 not found: ID does not exist"
Sep 16 14:04:50.671720 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.671695    2580 scope.go:117] "RemoveContainer" containerID="944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9"
Sep 16 14:04:50.680418 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.680395    2580 scope.go:117] "RemoveContainer" containerID="944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9"
Sep 16 14:04:50.680711 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:04:50.680691    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9\": container with ID starting with 944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9 not found: ID does not exist" containerID="944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9"
Sep 16 14:04:50.680761 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.680718    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9"} err="failed to get container status \"944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9\": rpc error: code = NotFound desc = could not find container \"944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9\": container with ID starting with 944d5c988d8db5761ffb33b79b5ee8ad5837fc895efb03dc79b9ba76d6bd81c9 not found: ID does not exist"
Sep 16 14:04:50.688053 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.688024    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"]
Sep 16 14:04:50.694098 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.694073    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2ed470-cdd98766-n4tlq"]
Sep 16 14:04:50.703162 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.703138    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"]
Sep 16 14:04:50.706506 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.706482    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2ed470-7774b96f4f-vld8l"]
Sep 16 14:04:50.803544 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.803508    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="088a34be-9f50-4d5b-aa0a-66bc790d18c4" path="/var/lib/kubelet/pods/088a34be-9f50-4d5b-aa0a-66bc790d18c4/volumes"
Sep 16 14:04:50.803942 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.803933    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a11498b4-a14c-4c68-bd87-750d9d7d5a0e" path="/var/lib/kubelet/pods/a11498b4-a14c-4c68-bd87-750d9d7d5a0e/volumes"
Sep 16 14:04:50.804281 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:50.804266    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8" path="/var/lib/kubelet/pods/f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8/volumes"
Sep 16 14:04:53.128527 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:53.128493    2580 scope.go:117] "RemoveContainer" containerID="527b396d44302c127285777dea10578de79138423583cd1dbcbe5f39a39bdb3e"
Sep 16 14:04:53.136615 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:04:53.136596    2580 scope.go:117] "RemoveContainer" containerID="1972b6984d89864c449811fa892df1049d18068d3e011f48f62b711637cc65d3"
Sep 16 14:05:38.544316 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544263    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"]
Sep 16 14:05:38.544749 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544645    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a11498b4-a14c-4c68-bd87-750d9d7d5a0e" containerName="payload-processing"
Sep 16 14:05:38.544749 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544659    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a11498b4-a14c-4c68-bd87-750d9d7d5a0e" containerName="payload-processing"
Sep 16 14:05:38.544749 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544676    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="088a34be-9f50-4d5b-aa0a-66bc790d18c4" containerName="payload-pre-processing"
Sep 16 14:05:38.544749 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544682    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="088a34be-9f50-4d5b-aa0a-66bc790d18c4" containerName="payload-pre-processing"
Sep 16 14:05:38.544749 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544690    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8" containerName="revoke-keys"
Sep 16 14:05:38.544749 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544695    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8" containerName="revoke-keys"
Sep 16 14:05:38.544960 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544753    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a11498b4-a14c-4c68-bd87-750d9d7d5a0e" containerName="payload-processing"
Sep 16 14:05:38.544960 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544764    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="088a34be-9f50-4d5b-aa0a-66bc790d18c4" containerName="payload-pre-processing"
Sep 16 14:05:38.544960 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.544774    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f69d86d1-dbf0-4de7-b076-f8f4acdfbbe8" containerName="revoke-keys"
Sep 16 14:05:38.547628 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.547612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"
Sep 16 14:05:38.549777 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.549755    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 14:05:38.549931 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.549781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:05:38.554732 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.554707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"]
Sep 16 14:05:38.613448 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.613414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389187a9-e9df-43fb-880e-298084cb4295-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw\" (UID: \"389187a9-e9df-43fb-880e-298084cb4295\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"
Sep 16 14:05:38.713985 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.713949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389187a9-e9df-43fb-880e-298084cb4295-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw\" (UID: \"389187a9-e9df-43fb-880e-298084cb4295\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"
Sep 16 14:05:38.714676 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.714646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389187a9-e9df-43fb-880e-298084cb4295-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw\" (UID: \"389187a9-e9df-43fb-880e-298084cb4295\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"
Sep 16 14:05:38.858680 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:38.858604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"
Sep 16 14:05:39.128930 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:39.128886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"]
Sep 16 14:05:39.132215 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:05:39.132179    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389187a9_e9df_43fb_880e_298084cb4295.slice/crio-3c8723d55e5bb210757b205497ac26dcb110e48162890f661307d1c1be30423f WatchSource:0}: Error finding container 3c8723d55e5bb210757b205497ac26dcb110e48162890f661307d1c1be30423f: Status 404 returned error can't find the container with id 3c8723d55e5bb210757b205497ac26dcb110e48162890f661307d1c1be30423f
Sep 16 14:05:39.896684 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:39.896651    2580 generic.go:356] "Generic (PLEG): container finished" podID="389187a9-e9df-43fb-880e-298084cb4295" containerID="b9ffe2b4ac917229223e0f2194166d0b4e0b506dc72a2aac83ca07e7626c6b66" exitCode=0
Sep 16 14:05:39.897072 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:39.896688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw" event={"ID":"389187a9-e9df-43fb-880e-298084cb4295","Type":"ContainerDied","Data":"b9ffe2b4ac917229223e0f2194166d0b4e0b506dc72a2aac83ca07e7626c6b66"}
Sep 16 14:05:39.897072 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:39.896730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw" event={"ID":"389187a9-e9df-43fb-880e-298084cb4295","Type":"ContainerStarted","Data":"3c8723d55e5bb210757b205497ac26dcb110e48162890f661307d1c1be30423f"}
Sep 16 14:05:41.029210 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:41.029187    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"
Sep 16 14:05:41.132486 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:41.132454    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389187a9-e9df-43fb-880e-298084cb4295-maas-api-service-ca\") pod \"389187a9-e9df-43fb-880e-298084cb4295\" (UID: \"389187a9-e9df-43fb-880e-298084cb4295\") "
Sep 16 14:05:41.132785 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:41.132762    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/389187a9-e9df-43fb-880e-298084cb4295-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "389187a9-e9df-43fb-880e-298084cb4295" (UID: "389187a9-e9df-43fb-880e-298084cb4295"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:05:41.233356 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:41.233296    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389187a9-e9df-43fb-880e-298084cb4295-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:05:41.907674 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:41.907644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw" event={"ID":"389187a9-e9df-43fb-880e-298084cb4295","Type":"ContainerDied","Data":"3c8723d55e5bb210757b205497ac26dcb110e48162890f661307d1c1be30423f"}
Sep 16 14:05:41.907834 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:41.907692    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c8723d55e5bb210757b205497ac26dcb110e48162890f661307d1c1be30423f"
Sep 16 14:05:41.907834 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:41.907652    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"
Sep 16 14:05:49.064118 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.064027    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"]
Sep 16 14:05:49.064518 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.064293    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5" podUID="98bc8ddb-9e5c-4be1-b48c-53168adda04f" containerName="payload-pre-processing" containerID="cri-o://3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203" gracePeriod=30
Sep 16 14:05:49.234962 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.234929    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"]
Sep 16 14:05:49.237580 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.237553    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9e8244-s9qkw"]
Sep 16 14:05:49.320997 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.320939    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:05:49.397310 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.397274    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98bc8ddb-9e5c-4be1-b48c-53168adda04f-plugins-config-volume\") pod \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") "
Sep 16 14:05:49.397484 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.397334    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxl9m\" (UniqueName: \"kubernetes.io/projected/98bc8ddb-9e5c-4be1-b48c-53168adda04f-kube-api-access-wxl9m\") pod \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\" (UID: \"98bc8ddb-9e5c-4be1-b48c-53168adda04f\") "
Sep 16 14:05:49.397755 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.397725    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98bc8ddb-9e5c-4be1-b48c-53168adda04f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98bc8ddb-9e5c-4be1-b48c-53168adda04f" (UID: "98bc8ddb-9e5c-4be1-b48c-53168adda04f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:05:49.399492 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.399464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98bc8ddb-9e5c-4be1-b48c-53168adda04f-kube-api-access-wxl9m" (OuterVolumeSpecName: "kube-api-access-wxl9m") pod "98bc8ddb-9e5c-4be1-b48c-53168adda04f" (UID: "98bc8ddb-9e5c-4be1-b48c-53168adda04f"). InnerVolumeSpecName "kube-api-access-wxl9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:05:49.498048 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.498013    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98bc8ddb-9e5c-4be1-b48c-53168adda04f-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:05:49.498048 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.498043    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxl9m\" (UniqueName: \"kubernetes.io/projected/98bc8ddb-9e5c-4be1-b48c-53168adda04f-kube-api-access-wxl9m\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:05:49.950620 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.950584    2580 generic.go:356] "Generic (PLEG): container finished" podID="98bc8ddb-9e5c-4be1-b48c-53168adda04f" containerID="3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203" exitCode=0
Sep 16 14:05:49.950843 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.950630    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"
Sep 16 14:05:49.950843 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.950685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5" event={"ID":"98bc8ddb-9e5c-4be1-b48c-53168adda04f","Type":"ContainerDied","Data":"3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203"}
Sep 16 14:05:49.950843 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.950745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5" event={"ID":"98bc8ddb-9e5c-4be1-b48c-53168adda04f","Type":"ContainerDied","Data":"5a49cd3f626b0535f7edba03c5a7c4311f264e75601cc49bb77010c05499e831"}
Sep 16 14:05:49.950843 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.950774    2580 scope.go:117] "RemoveContainer" containerID="3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203"
Sep 16 14:05:49.960074 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.960040    2580 scope.go:117] "RemoveContainer" containerID="3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203"
Sep 16 14:05:49.960382 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:05:49.960364    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203\": container with ID starting with 3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203 not found: ID does not exist" containerID="3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203"
Sep 16 14:05:49.960443 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.960392    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203"} err="failed to get container status \"3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203\": rpc error: code = NotFound desc = could not find container \"3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203\": container with ID starting with 3f52b9f117e6921abd573b5d3183665f121a637f8ea45edb8cd3c1b03e303203 not found: ID does not exist"
Sep 16 14:05:49.971874 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.971853    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"]
Sep 16 14:05:49.977803 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:49.977784    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9e8244-585f578586-xt4m5"]
Sep 16 14:05:50.801964 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:50.801925    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="389187a9-e9df-43fb-880e-298084cb4295" path="/var/lib/kubelet/pods/389187a9-e9df-43fb-880e-298084cb4295/volumes"
Sep 16 14:05:50.802487 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:50.802462    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98bc8ddb-9e5c-4be1-b48c-53168adda04f" path="/var/lib/kubelet/pods/98bc8ddb-9e5c-4be1-b48c-53168adda04f/volumes"
Sep 16 14:05:53.175198 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:53.175160    2580 scope.go:117] "RemoveContainer" containerID="a4c504c8e331bd41ffa73e8f7fa2cd96fc5f2c03282e7ac6f7e212c7fc986b64"
Sep 16 14:05:53.183428 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:53.183406    2580 scope.go:117] "RemoveContainer" containerID="9a7dd8182a0986c4fb20ca0f537a54234bcee7ef5ad036a66574fc4bf13cf244"
Sep 16 14:05:53.190533 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:53.190512    2580 scope.go:117] "RemoveContainer" containerID="5ac3a07ec90fdac7b15c9d8953098946000aaa2807d6601e6a0d0f95618560f8"
Sep 16 14:05:55.862522 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.862495    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"]
Sep 16 14:05:55.862924 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.862810    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98bc8ddb-9e5c-4be1-b48c-53168adda04f" containerName="payload-pre-processing"
Sep 16 14:05:55.862924 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.862821    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="98bc8ddb-9e5c-4be1-b48c-53168adda04f" containerName="payload-pre-processing"
Sep 16 14:05:55.862924 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.862838    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="389187a9-e9df-43fb-880e-298084cb4295" containerName="revoke-keys"
Sep 16 14:05:55.862924 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.862844    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="389187a9-e9df-43fb-880e-298084cb4295" containerName="revoke-keys"
Sep 16 14:05:55.862924 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.862890    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="389187a9-e9df-43fb-880e-298084cb4295" containerName="revoke-keys"
Sep 16 14:05:55.862924 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.862914    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="98bc8ddb-9e5c-4be1-b48c-53168adda04f" containerName="payload-pre-processing"
Sep 16 14:05:55.867585 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.867552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"
Sep 16 14:05:55.870797 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.870774    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 14:05:55.870980 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.870776    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:05:55.872596 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.872569    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"]
Sep 16 14:05:55.953054 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:55.953021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6bec50-d99b-46f4-9664-c0e7e61954a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr\" (UID: \"3f6bec50-d99b-46f4-9664-c0e7e61954a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"
Sep 16 14:05:56.054311 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:56.054276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6bec50-d99b-46f4-9664-c0e7e61954a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr\" (UID: \"3f6bec50-d99b-46f4-9664-c0e7e61954a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"
Sep 16 14:05:56.054932 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:56.054884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6bec50-d99b-46f4-9664-c0e7e61954a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr\" (UID: \"3f6bec50-d99b-46f4-9664-c0e7e61954a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"
Sep 16 14:05:56.179415 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:56.179384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"
Sep 16 14:05:56.449540 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:56.449423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"]
Sep 16 14:05:56.452345 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:05:56.452314    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6bec50_d99b_46f4_9664_c0e7e61954a5.slice/crio-45e5bf3c1cdbf5b5d80d92f1312ccafacac5f722da46d0bd9274c21b1eec60ac WatchSource:0}: Error finding container 45e5bf3c1cdbf5b5d80d92f1312ccafacac5f722da46d0bd9274c21b1eec60ac: Status 404 returned error can't find the container with id 45e5bf3c1cdbf5b5d80d92f1312ccafacac5f722da46d0bd9274c21b1eec60ac
Sep 16 14:05:56.985053 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:56.985015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr" event={"ID":"3f6bec50-d99b-46f4-9664-c0e7e61954a5","Type":"ContainerStarted","Data":"a08b3885bc9efd21d6faff774f43ab5ea2b77f7e1f11c6ff82d75f89c159e67f"}
Sep 16 14:05:56.985506 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:56.985080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr" event={"ID":"3f6bec50-d99b-46f4-9664-c0e7e61954a5","Type":"ContainerStarted","Data":"45e5bf3c1cdbf5b5d80d92f1312ccafacac5f722da46d0bd9274c21b1eec60ac"}
Sep 16 14:05:56.999048 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:56.998999    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr" podStartSLOduration=1.998983987 podStartE2EDuration="1.998983987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:05:56.997798547 +0000 UTC m=+1984.681456941" watchObservedRunningTime="2026-09-16 14:05:56.998983987 +0000 UTC m=+1984.682642420"
Sep 16 14:05:57.991672 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:57.991634    2580 generic.go:356] "Generic (PLEG): container finished" podID="3f6bec50-d99b-46f4-9664-c0e7e61954a5" containerID="a08b3885bc9efd21d6faff774f43ab5ea2b77f7e1f11c6ff82d75f89c159e67f" exitCode=0
Sep 16 14:05:57.992074 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:57.991722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr" event={"ID":"3f6bec50-d99b-46f4-9664-c0e7e61954a5","Type":"ContainerDied","Data":"a08b3885bc9efd21d6faff774f43ab5ea2b77f7e1f11c6ff82d75f89c159e67f"}
Sep 16 14:05:59.130260 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:59.130236    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"
Sep 16 14:05:59.182362 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:59.182333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6bec50-d99b-46f4-9664-c0e7e61954a5-maas-api-service-ca\") pod \"3f6bec50-d99b-46f4-9664-c0e7e61954a5\" (UID: \"3f6bec50-d99b-46f4-9664-c0e7e61954a5\") "
Sep 16 14:05:59.182690 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:59.182667    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f6bec50-d99b-46f4-9664-c0e7e61954a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f6bec50-d99b-46f4-9664-c0e7e61954a5" (UID: "3f6bec50-d99b-46f4-9664-c0e7e61954a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:05:59.283198 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:05:59.283110    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f6bec50-d99b-46f4-9664-c0e7e61954a5-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:00.007773 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:00.007736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr" event={"ID":"3f6bec50-d99b-46f4-9664-c0e7e61954a5","Type":"ContainerDied","Data":"45e5bf3c1cdbf5b5d80d92f1312ccafacac5f722da46d0bd9274c21b1eec60ac"}
Sep 16 14:06:00.007984 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:00.007792    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45e5bf3c1cdbf5b5d80d92f1312ccafacac5f722da46d0bd9274c21b1eec60ac"
Sep 16 14:06:00.007984 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:00.007750    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"
Sep 16 14:06:06.330929 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.330880    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"]
Sep 16 14:06:06.332656 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.332629    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2284bd-29826120-t2fwh"]
Sep 16 14:06:06.354094 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.354065    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"]
Sep 16 14:06:06.354352 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.354322    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w" podUID="ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" containerName="payload-processing" containerID="cri-o://86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36" gracePeriod=30
Sep 16 14:06:06.360920 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.360876    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"]
Sep 16 14:06:06.361111 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.361086    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n" podUID="49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" containerName="payload-pre-processing" containerID="cri-o://b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5" gracePeriod=30
Sep 16 14:06:06.512134 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.511972    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"]
Sep 16 14:06:06.513729 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.513701    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2284bd-9h4fr"]
Sep 16 14:06:06.629132 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.629053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 14:06:06.632377 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.632357    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 14:06:06.744792 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.744745    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns95t\" (UniqueName: \"kubernetes.io/projected/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-kube-api-access-ns95t\") pod \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") "
Sep 16 14:06:06.744792 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.744802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-plugins-config-volume\") pod \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") "
Sep 16 14:06:06.745048 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.744873    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-plugins-config-volume\") pod \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\" (UID: \"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c\") "
Sep 16 14:06:06.745048 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.744976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw4jt\" (UniqueName: \"kubernetes.io/projected/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-kube-api-access-gw4jt\") pod \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\" (UID: \"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6\") "
Sep 16 14:06:06.745279 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.745247    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" (UID: "49e16ec0-2f2b-4209-b9ac-f854d9c46e5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:06.745341 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.745266    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" (UID: "ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:06.747084 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.747054    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-kube-api-access-gw4jt" (OuterVolumeSpecName: "kube-api-access-gw4jt") pod "ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" (UID: "ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6"). InnerVolumeSpecName "kube-api-access-gw4jt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:06:06.747169 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.747150    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-kube-api-access-ns95t" (OuterVolumeSpecName: "kube-api-access-ns95t") pod "49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" (UID: "49e16ec0-2f2b-4209-b9ac-f854d9c46e5c"). InnerVolumeSpecName "kube-api-access-ns95t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:06:06.804828 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.804785    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f6bec50-d99b-46f4-9664-c0e7e61954a5" path="/var/lib/kubelet/pods/3f6bec50-d99b-46f4-9664-c0e7e61954a5/volumes"
Sep 16 14:06:06.805308 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.805282    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0" path="/var/lib/kubelet/pods/e183b128-d92b-4f2d-bf6b-ac0e8e2d2fd0/volumes"
Sep 16 14:06:06.845886 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.845855    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw4jt\" (UniqueName: \"kubernetes.io/projected/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-kube-api-access-gw4jt\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:06.845886 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.845879    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns95t\" (UniqueName: \"kubernetes.io/projected/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-kube-api-access-ns95t\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:06.845886 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.845891    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:06.846095 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:06.845912    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:07.040883 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.040846    2580 generic.go:356] "Generic (PLEG): container finished" podID="ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" containerID="86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36" exitCode=0
Sep 16 14:06:07.041099 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.040921    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"
Sep 16 14:06:07.041099 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.040935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w" event={"ID":"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6","Type":"ContainerDied","Data":"86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36"}
Sep 16 14:06:07.041099 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.040992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w" event={"ID":"ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6","Type":"ContainerDied","Data":"cfb163c70f1fc4625545e200787c3af55b682bdd9f8e3757931365c8f0f30c06"}
Sep 16 14:06:07.041099 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.041017    2580 scope.go:117] "RemoveContainer" containerID="86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36"
Sep 16 14:06:07.042352 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.042319    2580 generic.go:356] "Generic (PLEG): container finished" podID="49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" containerID="b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5" exitCode=0
Sep 16 14:06:07.042450 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.042352    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"
Sep 16 14:06:07.042450 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.042403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n" event={"ID":"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c","Type":"ContainerDied","Data":"b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5"}
Sep 16 14:06:07.042450 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.042437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n" event={"ID":"49e16ec0-2f2b-4209-b9ac-f854d9c46e5c","Type":"ContainerDied","Data":"1ab5ebd3a32fd0427280462a50752dbc331dcb35995bb37d12d6a1e9f50fdedd"}
Sep 16 14:06:07.049500 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.049486    2580 scope.go:117] "RemoveContainer" containerID="86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36"
Sep 16 14:06:07.049774 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:06:07.049756    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36\": container with ID starting with 86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36 not found: ID does not exist" containerID="86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36"
Sep 16 14:06:07.049845 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.049781    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36"} err="failed to get container status \"86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36\": rpc error: code = NotFound desc = could not find container \"86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36\": container with ID starting with 86ea3f35420b55581892e0b6bcbacc7f8ec9eb72abdde9a0f895f81a87ad4d36 not found: ID does not exist"
Sep 16 14:06:07.049845 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.049796    2580 scope.go:117] "RemoveContainer" containerID="b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5"
Sep 16 14:06:07.058407 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.058234    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"]
Sep 16 14:06:07.059337 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.059315    2580 scope.go:117] "RemoveContainer" containerID="b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5"
Sep 16 14:06:07.059889 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:06:07.059855    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5\": container with ID starting with b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5 not found: ID does not exist" containerID="b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5"
Sep 16 14:06:07.060039 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.059895    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5"} err="failed to get container status \"b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5\": rpc error: code = NotFound desc = could not find container \"b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5\": container with ID starting with b26a23ad2a4f5d8327068121d5fabd65f8eac88192c32cf20ff8780bd5679ba5 not found: ID does not exist"
Sep 16 14:06:07.062125 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.062104    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2284bd-96c548dff-s5z6w"]
Sep 16 14:06:07.070496 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.070473    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"]
Sep 16 14:06:07.072306 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:07.072286    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2284bd-6f88457598-msj6n"]
Sep 16 14:06:08.802092 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:08.802060    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" path="/var/lib/kubelet/pods/49e16ec0-2f2b-4209-b9ac-f854d9c46e5c/volumes"
Sep 16 14:06:08.802462 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:08.802400    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" path="/var/lib/kubelet/pods/ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6/volumes"
Sep 16 14:06:13.991922 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.991875    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"]
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992185    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" containerName="payload-pre-processing"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992196    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" containerName="payload-pre-processing"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992214    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" containerName="payload-processing"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992220    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" containerName="payload-processing"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992230    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6bec50-d99b-46f4-9664-c0e7e61954a5" containerName="revoke-keys"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992237    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6bec50-d99b-46f4-9664-c0e7e61954a5" containerName="revoke-keys"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992283    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e16ec0-2f2b-4209-b9ac-f854d9c46e5c" containerName="payload-pre-processing"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992291    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba6a7e9a-3d2b-435e-9dc9-e459ec176cb6" containerName="payload-processing"
Sep 16 14:06:13.992392 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.992299    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6bec50-d99b-46f4-9664-c0e7e61954a5" containerName="revoke-keys"
Sep 16 14:06:13.996604 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.996580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"
Sep 16 14:06:13.998578 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.998552    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:06:13.998737 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:13.998557    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmnnx\""
Sep 16 14:06:14.003296 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:14.003119    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"]
Sep 16 14:06:14.103650 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:14.103617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f\" (UID: \"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"
Sep 16 14:06:14.204540 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:14.204508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f\" (UID: \"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"
Sep 16 14:06:14.205226 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:14.205207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f\" (UID: \"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"
Sep 16 14:06:14.310205 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:14.310119    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"
Sep 16 14:06:14.578809 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:14.578568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"]
Sep 16 14:06:14.581482 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:06:14.581454    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3227ece_8bcd_40f4_b3d0_4a561a5e5ae5.slice/crio-906f5e7efaf1ed131bff745bc103fe403582a558733eb875eb496a7aae45085b WatchSource:0}: Error finding container 906f5e7efaf1ed131bff745bc103fe403582a558733eb875eb496a7aae45085b: Status 404 returned error can't find the container with id 906f5e7efaf1ed131bff745bc103fe403582a558733eb875eb496a7aae45085b
Sep 16 14:06:15.082272 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:15.082239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f" event={"ID":"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5","Type":"ContainerStarted","Data":"fefd3edc89180f1cdd3a22ca28db6c5768b5e43c89fd2f7595c8170c6b479e3b"}
Sep 16 14:06:15.082637 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:15.082293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f" event={"ID":"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5","Type":"ContainerStarted","Data":"906f5e7efaf1ed131bff745bc103fe403582a558733eb875eb496a7aae45085b"}
Sep 16 14:06:15.095536 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:15.095479    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f" podStartSLOduration=2.095460742 podStartE2EDuration="2.095460742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:06:15.094729081 +0000 UTC m=+2002.778387474" watchObservedRunningTime="2026-09-16 14:06:15.095460742 +0000 UTC m=+2002.779119136"
Sep 16 14:06:16.088085 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:16.088050    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5" containerID="fefd3edc89180f1cdd3a22ca28db6c5768b5e43c89fd2f7595c8170c6b479e3b" exitCode=0
Sep 16 14:06:16.088481 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:16.088142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f" event={"ID":"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5","Type":"ContainerDied","Data":"fefd3edc89180f1cdd3a22ca28db6c5768b5e43c89fd2f7595c8170c6b479e3b"}
Sep 16 14:06:17.219358 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:17.219328    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"
Sep 16 14:06:17.330627 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:17.330588    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5-maas-api-service-ca\") pod \"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5\" (UID: \"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5\") "
Sep 16 14:06:17.330990 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:17.330966    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5" (UID: "a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:17.431474 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:17.431441    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5-maas-api-service-ca\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:18.098965 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:18.098928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f" event={"ID":"a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5","Type":"ContainerDied","Data":"906f5e7efaf1ed131bff745bc103fe403582a558733eb875eb496a7aae45085b"}
Sep 16 14:06:18.098965 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:18.098948    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"
Sep 16 14:06:18.099174 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:18.098976    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="906f5e7efaf1ed131bff745bc103fe403582a558733eb875eb496a7aae45085b"
Sep 16 14:06:24.494062 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.494032    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"]
Sep 16 14:06:24.496499 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.496474    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ac8045-29826120-55bls"]
Sep 16 14:06:24.516352 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.516322    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"]
Sep 16 14:06:24.516612 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.516564    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46" podUID="c42fd23f-7c49-496d-9cf8-37ead6d79045" containerName="payload-processing" containerID="cri-o://760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc" gracePeriod=30
Sep 16 14:06:24.523938 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.523894    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"]
Sep 16 14:06:24.524156 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.524128    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp" podUID="ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" containerName="payload-pre-processing" containerID="cri-o://4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd" gracePeriod=30
Sep 16 14:06:24.681624 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.681579    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"]
Sep 16 14:06:24.685599 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.685570    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ac8045-d767f"]
Sep 16 14:06:24.791941 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.791917    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 14:06:24.795233 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.795206    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 14:06:24.802278 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.802252    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c4718e-6273-49a6-880c-ecdf453134b8" path="/var/lib/kubelet/pods/49c4718e-6273-49a6-880c-ecdf453134b8/volumes"
Sep 16 14:06:24.802583 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.802568    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5" path="/var/lib/kubelet/pods/a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5/volumes"
Sep 16 14:06:24.892244 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.892202    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sgjc\" (UniqueName: \"kubernetes.io/projected/c42fd23f-7c49-496d-9cf8-37ead6d79045-kube-api-access-9sgjc\") pod \"c42fd23f-7c49-496d-9cf8-37ead6d79045\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") "
Sep 16 14:06:24.892417 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.892348    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fc47q\" (UniqueName: \"kubernetes.io/projected/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-kube-api-access-fc47q\") pod \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") "
Sep 16 14:06:24.892417 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.892390    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c42fd23f-7c49-496d-9cf8-37ead6d79045-plugins-config-volume\") pod \"c42fd23f-7c49-496d-9cf8-37ead6d79045\" (UID: \"c42fd23f-7c49-496d-9cf8-37ead6d79045\") "
Sep 16 14:06:24.892542 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.892420    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-plugins-config-volume\") pod \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\" (UID: \"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d\") "
Sep 16 14:06:24.892839 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.892806    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c42fd23f-7c49-496d-9cf8-37ead6d79045-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c42fd23f-7c49-496d-9cf8-37ead6d79045" (UID: "c42fd23f-7c49-496d-9cf8-37ead6d79045"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:24.892979 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.892811    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" (UID: "ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:06:24.894489 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.894470    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-kube-api-access-fc47q" (OuterVolumeSpecName: "kube-api-access-fc47q") pod "ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" (UID: "ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d"). InnerVolumeSpecName "kube-api-access-fc47q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:06:24.894825 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.894800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42fd23f-7c49-496d-9cf8-37ead6d79045-kube-api-access-9sgjc" (OuterVolumeSpecName: "kube-api-access-9sgjc") pod "c42fd23f-7c49-496d-9cf8-37ead6d79045" (UID: "c42fd23f-7c49-496d-9cf8-37ead6d79045"). InnerVolumeSpecName "kube-api-access-9sgjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:06:24.993622 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.993582    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:24.993622 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.993622    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sgjc\" (UniqueName: \"kubernetes.io/projected/c42fd23f-7c49-496d-9cf8-37ead6d79045-kube-api-access-9sgjc\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:24.993815 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.993637    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc47q\" (UniqueName: \"kubernetes.io/projected/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d-kube-api-access-fc47q\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:24.993815 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:24.993652    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c42fd23f-7c49-496d-9cf8-37ead6d79045-plugins-config-volume\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:25.131008 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.130973    2580 generic.go:356] "Generic (PLEG): container finished" podID="c42fd23f-7c49-496d-9cf8-37ead6d79045" containerID="760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc" exitCode=0
Sep 16 14:06:25.131200 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.131023    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"
Sep 16 14:06:25.131200 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.131054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46" event={"ID":"c42fd23f-7c49-496d-9cf8-37ead6d79045","Type":"ContainerDied","Data":"760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc"}
Sep 16 14:06:25.131200 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.131112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46" event={"ID":"c42fd23f-7c49-496d-9cf8-37ead6d79045","Type":"ContainerDied","Data":"b4c665698e9aa203a2e2777d26feb36c978ff2d6440b7199f5a07e94b4b06ce5"}
Sep 16 14:06:25.131200 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.131145    2580 scope.go:117] "RemoveContainer" containerID="760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc"
Sep 16 14:06:25.132385 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.132366    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" containerID="4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd" exitCode=0
Sep 16 14:06:25.132488 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.132417    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"
Sep 16 14:06:25.132488 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.132438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp" event={"ID":"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d","Type":"ContainerDied","Data":"4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd"}
Sep 16 14:06:25.132488 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.132468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp" event={"ID":"ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d","Type":"ContainerDied","Data":"285be5d054e34ae47d239c829255ceec021a0b3b8d863cf2e3e923d84a837621"}
Sep 16 14:06:25.140253 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.140191    2580 scope.go:117] "RemoveContainer" containerID="760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc"
Sep 16 14:06:25.140518 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:06:25.140498    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc\": container with ID starting with 760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc not found: ID does not exist" containerID="760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc"
Sep 16 14:06:25.140598 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.140529    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc"} err="failed to get container status \"760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc\": rpc error: code = NotFound desc = could not find container \"760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc\": container with ID starting with 760790a38d307b344bd9a30dde05c9335157a723c4e3c404332d408be24a3dfc not found: ID does not exist"
Sep 16 14:06:25.140598 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.140550    2580 scope.go:117] "RemoveContainer" containerID="4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd"
Sep 16 14:06:25.148394 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.148372    2580 scope.go:117] "RemoveContainer" containerID="4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd"
Sep 16 14:06:25.148633 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:06:25.148613    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd\": container with ID starting with 4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd not found: ID does not exist" containerID="4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd"
Sep 16 14:06:25.148697 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.148639    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd"} err="failed to get container status \"4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd\": rpc error: code = NotFound desc = could not find container \"4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd\": container with ID starting with 4cb42d7be7708c5385a72e0d35fa8e64239590f739de28f29e7cfadb3cbd68fd not found: ID does not exist"
Sep 16 14:06:25.165560 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.165536    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"]
Sep 16 14:06:25.167916 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.167882    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ac8045-6b476b45f8-qtklp"]
Sep 16 14:06:25.177461 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.177437    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"]
Sep 16 14:06:25.181847 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:25.181827    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ac8045-8595b767f-vbr46"]
Sep 16 14:06:26.802496 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:26.802465    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c42fd23f-7c49-496d-9cf8-37ead6d79045" path="/var/lib/kubelet/pods/c42fd23f-7c49-496d-9cf8-37ead6d79045/volumes"
Sep 16 14:06:26.802859 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:26.802815    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" path="/var/lib/kubelet/pods/ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d/volumes"
Sep 16 14:06:34.239439 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.239405    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f96f875d-qwvrj"]
Sep 16 14:06:34.240132 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.239591    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f96f875d-qwvrj" podUID="23acd4b0-a3d3-4253-9a49-583af9bb8ddb" containerName="manager" containerID="cri-o://4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33" gracePeriod=30
Sep 16 14:06:34.483380 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.483355    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 14:06:34.574011 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.573925    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-kube-api-access-cqhtn\") pod \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") "
Sep 16 14:06:34.574011 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.573972    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-webhook-cert\") pod \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") "
Sep 16 14:06:34.574199 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.574025    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-metrics-certs\") pod \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\" (UID: \"23acd4b0-a3d3-4253-9a49-583af9bb8ddb\") "
Sep 16 14:06:34.576124 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.576091    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "23acd4b0-a3d3-4253-9a49-583af9bb8ddb" (UID: "23acd4b0-a3d3-4253-9a49-583af9bb8ddb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:06:34.576262 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.576154    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-kube-api-access-cqhtn" (OuterVolumeSpecName: "kube-api-access-cqhtn") pod "23acd4b0-a3d3-4253-9a49-583af9bb8ddb" (UID: "23acd4b0-a3d3-4253-9a49-583af9bb8ddb"). InnerVolumeSpecName "kube-api-access-cqhtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:06:34.576262 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.576167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "23acd4b0-a3d3-4253-9a49-583af9bb8ddb" (UID: "23acd4b0-a3d3-4253-9a49-583af9bb8ddb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:06:34.675216 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.675173    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:34.675216 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.675209    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-kube-api-access-cqhtn\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:34.675216 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:34.675218    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23acd4b0-a3d3-4253-9a49-583af9bb8ddb-webhook-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:06:35.181530 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.181490    2580 generic.go:356] "Generic (PLEG): container finished" podID="23acd4b0-a3d3-4253-9a49-583af9bb8ddb" containerID="4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33" exitCode=0
Sep 16 14:06:35.181708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.181542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-qwvrj" event={"ID":"23acd4b0-a3d3-4253-9a49-583af9bb8ddb","Type":"ContainerDied","Data":"4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33"}
Sep 16 14:06:35.181708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.181554    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-qwvrj"
Sep 16 14:06:35.181708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.181596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-qwvrj" event={"ID":"23acd4b0-a3d3-4253-9a49-583af9bb8ddb","Type":"ContainerDied","Data":"2407987c16b86491ce53799acc3d24f9362fc694f7217aec1e2b2c5d9185454e"}
Sep 16 14:06:35.181708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.181626    2580 scope.go:117] "RemoveContainer" containerID="4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33"
Sep 16 14:06:35.190928 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.190883    2580 scope.go:117] "RemoveContainer" containerID="4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33"
Sep 16 14:06:35.191232 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:06:35.191211    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33\": container with ID starting with 4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33 not found: ID does not exist" containerID="4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33"
Sep 16 14:06:35.191298 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.191240    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33"} err="failed to get container status \"4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33\": rpc error: code = NotFound desc = could not find container \"4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33\": container with ID starting with 4bc25de3e660e07e227d6c9ebe4f12983dd2a25d7447d8fc38813bb302969b33 not found: ID does not exist"
Sep 16 14:06:35.195455 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.195432    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f96f875d-qwvrj"]
Sep 16 14:06:35.198964 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:35.198942    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f96f875d-qwvrj"]
Sep 16 14:06:36.801269 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:36.801233    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23acd4b0-a3d3-4253-9a49-583af9bb8ddb" path="/var/lib/kubelet/pods/23acd4b0-a3d3-4253-9a49-583af9bb8ddb/volumes"
Sep 16 14:06:53.218651 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:53.218610    2580 scope.go:117] "RemoveContainer" containerID="0595d56b1816445adfe775d950cd7924cb027a5eaa267645c2d7d34b72a84115"
Sep 16 14:06:53.227662 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:06:53.227626    2580 scope.go:117] "RemoveContainer" containerID="0451efe1bef909c640edcd1af9943f83efc540d0c9916a162808c9ac69576695"
Sep 16 14:07:36.011257 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011223    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f96f875d-pfgzq"]
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011613    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5" containerName="revoke-keys"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011630    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5" containerName="revoke-keys"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011644    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" containerName="payload-pre-processing"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011655    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" containerName="payload-pre-processing"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011677    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23acd4b0-a3d3-4253-9a49-583af9bb8ddb" containerName="manager"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011686    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="23acd4b0-a3d3-4253-9a49-583af9bb8ddb" containerName="manager"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011702    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42fd23f-7c49-496d-9cf8-37ead6d79045" containerName="payload-processing"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011711    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42fd23f-7c49-496d-9cf8-37ead6d79045" containerName="payload-processing"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011786    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="23acd4b0-a3d3-4253-9a49-583af9bb8ddb" containerName="manager"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011798    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3227ece-8bcd-40f4-b3d0-4a561a5e5ae5" containerName="revoke-keys"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011810    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42fd23f-7c49-496d-9cf8-37ead6d79045" containerName="payload-processing"
Sep 16 14:07:36.011841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.011822    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee2f3a80-d462-4fa1-b3b8-5fb100a67b9d" containerName="payload-pre-processing"
Sep 16 14:07:36.014806 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.014784    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.017846 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.017814    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m6pl2\""
Sep 16 14:07:36.017994 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.017877    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 14:07:36.017994 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.017890    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 14:07:36.017994 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.017817    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 14:07:36.022945 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.022916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f96f875d-pfgzq"]
Sep 16 14:07:36.161732 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.161691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-webhook-cert\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.161732 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.161732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwhdw\" (UniqueName: \"kubernetes.io/projected/3ed1defd-5cdd-44bd-96b9-979ba4944c87-kube-api-access-vwhdw\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.161977 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.161850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-metrics-certs\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.263122 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.263025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwhdw\" (UniqueName: \"kubernetes.io/projected/3ed1defd-5cdd-44bd-96b9-979ba4944c87-kube-api-access-vwhdw\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.263270 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.263131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-metrics-certs\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.263270 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.263154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-webhook-cert\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.265629 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.265597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-metrics-certs\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.265629 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.265625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-webhook-cert\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.270245 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.270223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwhdw\" (UniqueName: \"kubernetes.io/projected/3ed1defd-5cdd-44bd-96b9-979ba4944c87-kube-api-access-vwhdw\") pod \"maas-controller-7f96f875d-pfgzq\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") " pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.326267 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.326231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:36.596469 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.596387    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f96f875d-pfgzq"]
Sep 16 14:07:36.599492 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:07:36.599466    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed1defd_5cdd_44bd_96b9_979ba4944c87.slice/crio-e44dd6b963d0dd2c5e1c93072de67dd2eb1cde151cff911d5a02db04a8b6c90d WatchSource:0}: Error finding container e44dd6b963d0dd2c5e1c93072de67dd2eb1cde151cff911d5a02db04a8b6c90d: Status 404 returned error can't find the container with id e44dd6b963d0dd2c5e1c93072de67dd2eb1cde151cff911d5a02db04a8b6c90d
Sep 16 14:07:36.600874 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:36.600854    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:07:37.454132 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:37.454029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-pfgzq" event={"ID":"3ed1defd-5cdd-44bd-96b9-979ba4944c87","Type":"ContainerStarted","Data":"06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c"}
Sep 16 14:07:37.454132 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:37.454088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-pfgzq" event={"ID":"3ed1defd-5cdd-44bd-96b9-979ba4944c87","Type":"ContainerStarted","Data":"e44dd6b963d0dd2c5e1c93072de67dd2eb1cde151cff911d5a02db04a8b6c90d"}
Sep 16 14:07:37.454132 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:37.454115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:37.491164 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:37.491113    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f96f875d-pfgzq" podStartSLOduration=1.939769011 podStartE2EDuration="2.491093474s" totalImagesPullingTime="551.324463ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:07:37.489027418 +0000 UTC m=+2085.172685824" watchObservedRunningTime="2026-09-16 14:07:37.491093474 +0000 UTC m=+2085.174751871"
Sep 16 14:07:48.460710 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:48.460672    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:07:52.844820 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:52.844787    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 14:07:52.846144 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:52.846118    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 14:07:59.938945 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:59.938893    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f96f875d-pfgzq"]
Sep 16 14:07:59.939408 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:07:59.939117    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f96f875d-pfgzq" podUID="3ed1defd-5cdd-44bd-96b9-979ba4944c87" containerName="manager" containerID="cri-o://06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c" gracePeriod=30
Sep 16 14:08:00.179326 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.179300    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:08:00.255669 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.255579    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwhdw\" (UniqueName: \"kubernetes.io/projected/3ed1defd-5cdd-44bd-96b9-979ba4944c87-kube-api-access-vwhdw\") pod \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") "
Sep 16 14:08:00.255669 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.255621    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-metrics-certs\") pod \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") "
Sep 16 14:08:00.255909 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.255692    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-webhook-cert\") pod \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\" (UID: \"3ed1defd-5cdd-44bd-96b9-979ba4944c87\") "
Sep 16 14:08:00.257708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.257675    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3ed1defd-5cdd-44bd-96b9-979ba4944c87" (UID: "3ed1defd-5cdd-44bd-96b9-979ba4944c87"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:08:00.257815 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.257765    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ed1defd-5cdd-44bd-96b9-979ba4944c87-kube-api-access-vwhdw" (OuterVolumeSpecName: "kube-api-access-vwhdw") pod "3ed1defd-5cdd-44bd-96b9-979ba4944c87" (UID: "3ed1defd-5cdd-44bd-96b9-979ba4944c87"). InnerVolumeSpecName "kube-api-access-vwhdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:08:00.257815 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.257770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3ed1defd-5cdd-44bd-96b9-979ba4944c87" (UID: "3ed1defd-5cdd-44bd-96b9-979ba4944c87"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:08:00.356630 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.356590    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-webhook-cert\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:08:00.356630 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.356627    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwhdw\" (UniqueName: \"kubernetes.io/projected/3ed1defd-5cdd-44bd-96b9-979ba4944c87-kube-api-access-vwhdw\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:08:00.356849 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.356643    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ed1defd-5cdd-44bd-96b9-979ba4944c87-metrics-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:08:00.558842 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.558749    2580 generic.go:356] "Generic (PLEG): container finished" podID="3ed1defd-5cdd-44bd-96b9-979ba4944c87" containerID="06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c" exitCode=0
Sep 16 14:08:00.558842 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.558807    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-pfgzq"
Sep 16 14:08:00.559077 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.558836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-pfgzq" event={"ID":"3ed1defd-5cdd-44bd-96b9-979ba4944c87","Type":"ContainerDied","Data":"06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c"}
Sep 16 14:08:00.559077 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.558930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-pfgzq" event={"ID":"3ed1defd-5cdd-44bd-96b9-979ba4944c87","Type":"ContainerDied","Data":"e44dd6b963d0dd2c5e1c93072de67dd2eb1cde151cff911d5a02db04a8b6c90d"}
Sep 16 14:08:00.559077 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.558953    2580 scope.go:117] "RemoveContainer" containerID="06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c"
Sep 16 14:08:00.567486 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.567469    2580 scope.go:117] "RemoveContainer" containerID="06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c"
Sep 16 14:08:00.567774 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:08:00.567753    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c\": container with ID starting with 06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c not found: ID does not exist" containerID="06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c"
Sep 16 14:08:00.567864 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.567785    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c"} err="failed to get container status \"06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c\": rpc error: code = NotFound desc = could not find container \"06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c\": container with ID starting with 06ed8167e74e912a2c8213460865472fbd01778414bc22111dfbe796345cb03c not found: ID does not exist"
Sep 16 14:08:00.580684 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.580660    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f96f875d-pfgzq"]
Sep 16 14:08:00.584118 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.584096    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f96f875d-pfgzq"]
Sep 16 14:08:00.803369 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:00.803329    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ed1defd-5cdd-44bd-96b9-979ba4944c87" path="/var/lib/kubelet/pods/3ed1defd-5cdd-44bd-96b9-979ba4944c87/volumes"
Sep 16 14:08:01.350383 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.350345    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f96f875d-sxmgg"]
Sep 16 14:08:01.350875 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.350763    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed1defd-5cdd-44bd-96b9-979ba4944c87" containerName="manager"
Sep 16 14:08:01.350875 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.350782    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed1defd-5cdd-44bd-96b9-979ba4944c87" containerName="manager"
Sep 16 14:08:01.350875 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.350860    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed1defd-5cdd-44bd-96b9-979ba4944c87" containerName="manager"
Sep 16 14:08:01.355318 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.355297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.357367 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.357349    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m6pl2\""
Sep 16 14:08:01.357864 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.357847    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 14:08:01.357977 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.357877    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 14:08:01.358066 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.358050    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 14:08:01.364264 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.364241    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f96f875d-sxmgg"]
Sep 16 14:08:01.466618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.466579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-metrics-certs\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.466618 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.466624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxmbd\" (UniqueName: \"kubernetes.io/projected/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-kube-api-access-rxmbd\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.466853 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.466715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-webhook-cert\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.567790 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.567756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxmbd\" (UniqueName: \"kubernetes.io/projected/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-kube-api-access-rxmbd\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.568000 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.567807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-webhook-cert\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.568000 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.567840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-metrics-certs\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.570302 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.570278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-webhook-cert\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.570410 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.570386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-metrics-certs\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.574882 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.574859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxmbd\" (UniqueName: \"kubernetes.io/projected/ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49-kube-api-access-rxmbd\") pod \"maas-controller-7f96f875d-sxmgg\" (UID: \"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49\") " pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.665977 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.665941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:01.935860 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:01.935780    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f96f875d-sxmgg"]
Sep 16 14:08:01.939025 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:08:01.938997    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf48b4f_75ba_4e92_95f6_15f1bdfc6e49.slice/crio-a8630ed2388e0d1adb4eeccb00cbe8809c8f638074dd6a319b784e1899b6a169 WatchSource:0}: Error finding container a8630ed2388e0d1adb4eeccb00cbe8809c8f638074dd6a319b784e1899b6a169: Status 404 returned error can't find the container with id a8630ed2388e0d1adb4eeccb00cbe8809c8f638074dd6a319b784e1899b6a169
Sep 16 14:08:02.571403 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:02.571290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-sxmgg" event={"ID":"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49","Type":"ContainerStarted","Data":"fce52d0498b0e0b2cc6934b89461730574e5c4bcbd3c354e84cc702cc0bbeda4"}
Sep 16 14:08:02.571403 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:02.571370    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:08:02.571403 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:02.571396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f96f875d-sxmgg" event={"ID":"ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49","Type":"ContainerStarted","Data":"a8630ed2388e0d1adb4eeccb00cbe8809c8f638074dd6a319b784e1899b6a169"}
Sep 16 14:08:02.586820 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:02.586772    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f96f875d-sxmgg" podStartSLOduration=1.222717921 podStartE2EDuration="1.586754889s" totalImagesPullingTime="364.036968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:02.585888502 +0000 UTC m=+2110.269546899" watchObservedRunningTime="2026-09-16 14:08:02.586754889 +0000 UTC m=+2110.270413311"
Sep 16 14:08:13.577829 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:08:13.577797    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f96f875d-sxmgg"
Sep 16 14:10:53.292014 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:10:53.291975    2580 scope.go:117] "RemoveContainer" containerID="c4e110d7d8a7dd5f8ccb79a6907d1a84b43283480ed755c8f1435157f9450729"
Sep 16 14:10:53.300106 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:10:53.300093    2580 scope.go:117] "RemoveContainer" containerID="532668f4ab68394cc548dbce885dd3302c6adfebf9090cbf411195f296d2d4b7"
Sep 16 14:11:07.149357 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.149320    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"]
Sep 16 14:11:07.152600 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.152582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.154697 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.154663    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-10275f33-kserve-self-signed-certs\""
Sep 16 14:11:07.160793 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.160764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"]
Sep 16 14:11:07.262267 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.262225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-dshm\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.262483 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.262277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f228b\" (UniqueName: \"kubernetes.io/projected/9519dad9-d0e1-4043-8765-dfec11bc69bb-kube-api-access-f228b\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.262483 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.262367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-home\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.262483 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.262402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9519dad9-d0e1-4043-8765-dfec11bc69bb-tls-certs\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.262483 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.262475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-model-cache\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.262811 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.262505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-tmp-dir\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363366 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f228b\" (UniqueName: \"kubernetes.io/projected/9519dad9-d0e1-4043-8765-dfec11bc69bb-kube-api-access-f228b\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363366 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-home\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363623 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9519dad9-d0e1-4043-8765-dfec11bc69bb-tls-certs\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363623 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-model-cache\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363623 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-tmp-dir\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363623 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-dshm\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363840 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-tmp-dir\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363840 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-home\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.363968 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.363837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-model-cache\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.365809 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.365785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-dshm\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.366124 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.366105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9519dad9-d0e1-4043-8765-dfec11bc69bb-tls-certs\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.370998 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.370978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f228b\" (UniqueName: \"kubernetes.io/projected/9519dad9-d0e1-4043-8765-dfec11bc69bb-kube-api-access-f228b\") pod \"e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") " pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.462759 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.462675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:07.729862 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:07.729785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"]
Sep 16 14:11:07.732835 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:11:07.732806    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9519dad9_d0e1_4043_8765_dfec11bc69bb.slice/crio-26d41771a3a9f8d8e59868254c23c53d0cfaf17b958cb45811dad1229ac0efdc WatchSource:0}: Error finding container 26d41771a3a9f8d8e59868254c23c53d0cfaf17b958cb45811dad1229ac0efdc: Status 404 returned error can't find the container with id 26d41771a3a9f8d8e59868254c23c53d0cfaf17b958cb45811dad1229ac0efdc
Sep 16 14:11:08.361509 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:08.361475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t" event={"ID":"9519dad9-d0e1-4043-8765-dfec11bc69bb","Type":"ContainerStarted","Data":"f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1"}
Sep 16 14:11:08.361509 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:08.361516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t" event={"ID":"9519dad9-d0e1-4043-8765-dfec11bc69bb","Type":"ContainerStarted","Data":"26d41771a3a9f8d8e59868254c23c53d0cfaf17b958cb45811dad1229ac0efdc"}
Sep 16 14:11:08.377684 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:08.377633    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t" podStartSLOduration=1.377617409 podStartE2EDuration="1.377617409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:11:08.375948878 +0000 UTC m=+2296.059607273" watchObservedRunningTime="2026-09-16 14:11:08.377617409 +0000 UTC m=+2296.061275804"
Sep 16 14:11:17.463209 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:17.463167    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:17.463597 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:17.463230    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:17.475352 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:17.475325    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:18.415145 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:18.415112    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:37.825866 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:37.825797    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"]
Sep 16 14:11:52.847708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:52.847608    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t" podUID="9519dad9-d0e1-4043-8765-dfec11bc69bb" containerName="main" containerID="cri-o://f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1" gracePeriod=45
Sep 16 14:11:53.086437 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.086414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:53.214498 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214476    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-dshm\") pod \"9519dad9-d0e1-4043-8765-dfec11bc69bb\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") "
Sep 16 14:11:53.214631 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214513    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9519dad9-d0e1-4043-8765-dfec11bc69bb-tls-certs\") pod \"9519dad9-d0e1-4043-8765-dfec11bc69bb\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") "
Sep 16 14:11:53.214631 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-home\") pod \"9519dad9-d0e1-4043-8765-dfec11bc69bb\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") "
Sep 16 14:11:53.214631 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-tmp-dir\") pod \"9519dad9-d0e1-4043-8765-dfec11bc69bb\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") "
Sep 16 14:11:53.214792 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214637    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f228b\" (UniqueName: \"kubernetes.io/projected/9519dad9-d0e1-4043-8765-dfec11bc69bb-kube-api-access-f228b\") pod \"9519dad9-d0e1-4043-8765-dfec11bc69bb\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") "
Sep 16 14:11:53.214792 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214673    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-model-cache\") pod \"9519dad9-d0e1-4043-8765-dfec11bc69bb\" (UID: \"9519dad9-d0e1-4043-8765-dfec11bc69bb\") "
Sep 16 14:11:53.214928 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9519dad9-d0e1-4043-8765-dfec11bc69bb" (UID: "9519dad9-d0e1-4043-8765-dfec11bc69bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:11:53.214928 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.214872    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-home" (OuterVolumeSpecName: "home") pod "9519dad9-d0e1-4043-8765-dfec11bc69bb" (UID: "9519dad9-d0e1-4043-8765-dfec11bc69bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:11:53.215047 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.215002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "9519dad9-d0e1-4043-8765-dfec11bc69bb" (UID: "9519dad9-d0e1-4043-8765-dfec11bc69bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:11:53.215047 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.215010    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-home\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:11:53.215047 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.215036    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-tmp-dir\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:11:53.216746 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.216720    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9519dad9-d0e1-4043-8765-dfec11bc69bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9519dad9-d0e1-4043-8765-dfec11bc69bb" (UID: "9519dad9-d0e1-4043-8765-dfec11bc69bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:11:53.216855 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.216771    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-dshm" (OuterVolumeSpecName: "dshm") pod "9519dad9-d0e1-4043-8765-dfec11bc69bb" (UID: "9519dad9-d0e1-4043-8765-dfec11bc69bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:11:53.216855 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.216844    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9519dad9-d0e1-4043-8765-dfec11bc69bb-kube-api-access-f228b" (OuterVolumeSpecName: "kube-api-access-f228b") pod "9519dad9-d0e1-4043-8765-dfec11bc69bb" (UID: "9519dad9-d0e1-4043-8765-dfec11bc69bb"). InnerVolumeSpecName "kube-api-access-f228b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:11:53.315853 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.315833    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9519dad9-d0e1-4043-8765-dfec11bc69bb-tls-certs\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:11:53.315853 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.315854    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f228b\" (UniqueName: \"kubernetes.io/projected/9519dad9-d0e1-4043-8765-dfec11bc69bb-kube-api-access-f228b\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:11:53.316037 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.315868    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-model-cache\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:11:53.316037 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.315880    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9519dad9-d0e1-4043-8765-dfec11bc69bb-dshm\") on node \"ip-10-0-130-207.ec2.internal\" DevicePath \"\""
Sep 16 14:11:53.319330 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.319312    2580 scope.go:117] "RemoveContainer" containerID="b9ffe2b4ac917229223e0f2194166d0b4e0b506dc72a2aac83ca07e7626c6b66"
Sep 16 14:11:53.547731 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.547662    2580 generic.go:356] "Generic (PLEG): container finished" podID="9519dad9-d0e1-4043-8765-dfec11bc69bb" containerID="f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1" exitCode=0
Sep 16 14:11:53.547731 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.547709    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"
Sep 16 14:11:53.547876 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.547712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t" event={"ID":"9519dad9-d0e1-4043-8765-dfec11bc69bb","Type":"ContainerDied","Data":"f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1"}
Sep 16 14:11:53.547876 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.547817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t" event={"ID":"9519dad9-d0e1-4043-8765-dfec11bc69bb","Type":"ContainerDied","Data":"26d41771a3a9f8d8e59868254c23c53d0cfaf17b958cb45811dad1229ac0efdc"}
Sep 16 14:11:53.547876 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.547838    2580 scope.go:117] "RemoveContainer" containerID="f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1"
Sep 16 14:11:53.555789 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.555772    2580 scope.go:117] "RemoveContainer" containerID="f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1"
Sep 16 14:11:53.556070 ip-10-0-130-207 kubenswrapper[2580]: E0916 14:11:53.556047    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1\": container with ID starting with f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1 not found: ID does not exist" containerID="f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1"
Sep 16 14:11:53.556156 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.556075    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1"} err="failed to get container status \"f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1\": rpc error: code = NotFound desc = could not find container \"f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1\": container with ID starting with f058eea552c716342c9a6e2bbb11aa910ac141219088a65da088c564362d4ca1 not found: ID does not exist"
Sep 16 14:11:53.566987 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.566965    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"]
Sep 16 14:11:53.571325 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:53.571302    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-10275f33-kserve-5fc6c87c75-fqs8t"]
Sep 16 14:11:54.801916 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:11:54.801871    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9519dad9-d0e1-4043-8765-dfec11bc69bb" path="/var/lib/kubelet/pods/9519dad9-d0e1-4043-8765-dfec11bc69bb/volumes"
Sep 16 14:12:27.347874 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:27.347839    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65b68fcbc9-njr2t_ed40d17e-2d69-4125-abdf-6f95c4a4e06a/authorino/0.log"
Sep 16 14:12:33.185017 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:33.184986    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f96f875d-sxmgg_ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49/manager/0.log"
Sep 16 14:12:33.669272 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:33.669244    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6444576fdb-62lqn_6abfc89e-1c1e-4c68-84dd-f0cbd985a965/copy-manifests/0.log"
Sep 16 14:12:33.678308 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:33.678285    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6444576fdb-62lqn_6abfc89e-1c1e-4c68-84dd-f0cbd985a965/manager/0.log"
Sep 16 14:12:35.445641 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:35.445610    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65b68fcbc9-njr2t_ed40d17e-2d69-4125-abdf-6f95c4a4e06a/authorino/0.log"
Sep 16 14:12:35.789882 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:35.789803    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-cjj6h_534957fb-aaa7-4f63-8ae1-d2f55a1869b7/kuadrant-console-plugin/0.log"
Sep 16 14:12:36.879708 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:36.879672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bbd697f4-l4hhd_8ce32683-d750-4f31-888c-3db9ef3e21f1/kube-auth-proxy/0.log"
Sep 16 14:12:37.125364 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:37.125337    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-pzqqj_3b109d40-55e2-4176-81f4-28f9197df298/payload-pre-processing/0.log"
Sep 16 14:12:37.711681 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:37.711652    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rvtc4_ecc22a7d-6d9f-493e-9aa1-9d8edcb98690/main/0.log"
Sep 16 14:12:38.162626 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:38.162593    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgjx6f_cd6e05bd-e925-4574-8f68-7ee265162c55/main/0.log"
Sep 16 14:12:38.272873 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:38.272852    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hwv5c_03c48375-73d3-40ed-bd85-9996f0995c5f/main/0.log"
Sep 16 14:12:43.549858 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.549824    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6t22r/must-gather-rmlbh"]
Sep 16 14:12:43.550238 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.550126    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9519dad9-d0e1-4043-8765-dfec11bc69bb" containerName="main"
Sep 16 14:12:43.550238 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.550138    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9519dad9-d0e1-4043-8765-dfec11bc69bb" containerName="main"
Sep 16 14:12:43.550238 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.550189    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9519dad9-d0e1-4043-8765-dfec11bc69bb" containerName="main"
Sep 16 14:12:43.553072 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.553052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:43.555948 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.555925    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6t22r\"/\"kube-root-ca.crt\""
Sep 16 14:12:43.556063 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.555975    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6t22r\"/\"openshift-service-ca.crt\""
Sep 16 14:12:43.556463 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.556448    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6t22r\"/\"default-dockercfg-qntd4\""
Sep 16 14:12:43.562680 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.562662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6t22r/must-gather-rmlbh"]
Sep 16 14:12:43.673687 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.673656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq5p9\" (UniqueName: \"kubernetes.io/projected/c699a259-170e-4d40-8bde-56394c66a350-kube-api-access-lq5p9\") pod \"must-gather-rmlbh\" (UID: \"c699a259-170e-4d40-8bde-56394c66a350\") " pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:43.673841 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.673697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c699a259-170e-4d40-8bde-56394c66a350-must-gather-output\") pod \"must-gather-rmlbh\" (UID: \"c699a259-170e-4d40-8bde-56394c66a350\") " pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:43.774541 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.774513    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c699a259-170e-4d40-8bde-56394c66a350-must-gather-output\") pod \"must-gather-rmlbh\" (UID: \"c699a259-170e-4d40-8bde-56394c66a350\") " pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:43.774697 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.774603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq5p9\" (UniqueName: \"kubernetes.io/projected/c699a259-170e-4d40-8bde-56394c66a350-kube-api-access-lq5p9\") pod \"must-gather-rmlbh\" (UID: \"c699a259-170e-4d40-8bde-56394c66a350\") " pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:43.774842 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.774823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c699a259-170e-4d40-8bde-56394c66a350-must-gather-output\") pod \"must-gather-rmlbh\" (UID: \"c699a259-170e-4d40-8bde-56394c66a350\") " pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:43.782199 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.782175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq5p9\" (UniqueName: \"kubernetes.io/projected/c699a259-170e-4d40-8bde-56394c66a350-kube-api-access-lq5p9\") pod \"must-gather-rmlbh\" (UID: \"c699a259-170e-4d40-8bde-56394c66a350\") " pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:43.862840 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:43.862792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6t22r/must-gather-rmlbh"
Sep 16 14:12:44.129637 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:44.129610    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6t22r/must-gather-rmlbh"]
Sep 16 14:12:44.133783 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:12:44.133760    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc699a259_170e_4d40_8bde_56394c66a350.slice/crio-0403743be77fb7090cf742af5499377f6d7f80f625543629608915a7a4a32e2c WatchSource:0}: Error finding container 0403743be77fb7090cf742af5499377f6d7f80f625543629608915a7a4a32e2c: Status 404 returned error can't find the container with id 0403743be77fb7090cf742af5499377f6d7f80f625543629608915a7a4a32e2c
Sep 16 14:12:44.135588 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:44.135569    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:12:44.779302 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:44.779261    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6t22r/must-gather-rmlbh" event={"ID":"c699a259-170e-4d40-8bde-56394c66a350","Type":"ContainerStarted","Data":"0403743be77fb7090cf742af5499377f6d7f80f625543629608915a7a4a32e2c"}
Sep 16 14:12:45.792757 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:45.792671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6t22r/must-gather-rmlbh" event={"ID":"c699a259-170e-4d40-8bde-56394c66a350","Type":"ContainerStarted","Data":"e9008f97f5cc9f5256aabd28c1d2611aa17a16e928f3a15e8e7d0d91fbbb72a9"}
Sep 16 14:12:45.792757 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:45.792726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6t22r/must-gather-rmlbh" event={"ID":"c699a259-170e-4d40-8bde-56394c66a350","Type":"ContainerStarted","Data":"df82a6c718c74c1bdbecddc11290e9770455a6764014b5da700833b95c5d670b"}
Sep 16 14:12:45.810157 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:45.810096    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6t22r/must-gather-rmlbh" podStartSLOduration=2.127735181 podStartE2EDuration="2.810075016s" totalImagesPullingTime="682.339835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:45.80759251 +0000 UTC m=+2393.491250900" watchObservedRunningTime="2026-09-16 14:12:45.810075016 +0000 UTC m=+2393.493733412"
Sep 16 14:12:46.376441 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:46.376408    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sxq2j_78390bc8-31f3-4f46-81cb-08992be7587b/global-pull-secret-syncer/0.log"
Sep 16 14:12:46.450441 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:46.450404    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vj42r_4280dfa4-c6ca-4824-8d64-e4d90bf2f176/konnectivity-agent/0.log"
Sep 16 14:12:46.502210 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:46.502182    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-207.ec2.internal_4054c0090de8f54aed802a5d555be580/haproxy/0.log"
Sep 16 14:12:51.567425 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:51.567382    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65b68fcbc9-njr2t_ed40d17e-2d69-4125-abdf-6f95c4a4e06a/authorino/0.log"
Sep 16 14:12:51.650105 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:51.650075    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-cjj6h_534957fb-aaa7-4f63-8ae1-d2f55a1869b7/kuadrant-console-plugin/0.log"
Sep 16 14:12:52.894664 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:52.894618    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 14:12:52.907922 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:52.906929    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5vh56_50365d01-e172-4f8f-bd73-3f62082b0cf1/ovn-acl-logging/0.log"
Sep 16 14:12:53.337067 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:53.336985    2580 scope.go:117] "RemoveContainer" containerID="fefd3edc89180f1cdd3a22ca28db6c5768b5e43c89fd2f7595c8170c6b479e3b"
Sep 16 14:12:53.351514 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:53.351444    2580 scope.go:117] "RemoveContainer" containerID="a08b3885bc9efd21d6faff774f43ab5ea2b77f7e1f11c6ff82d75f89c159e67f"
Sep 16 14:12:53.516324 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:53.516291    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h9fxb_d0abaff1-c5a8-4893-9303-2aa90e7af317/node-exporter/0.log"
Sep 16 14:12:53.537862 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:53.537825    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h9fxb_d0abaff1-c5a8-4893-9303-2aa90e7af317/kube-rbac-proxy/0.log"
Sep 16 14:12:53.585951 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:53.585917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h9fxb_d0abaff1-c5a8-4893-9303-2aa90e7af317/init-textfile/0.log"
Sep 16 14:12:57.089228 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.089187    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"]
Sep 16 14:12:57.093380 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.093346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.103122 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.103100    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"]
Sep 16 14:12:57.201176 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.201133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-lib-modules\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.201496 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.201475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-podres\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.201694 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.201675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqlzw\" (UniqueName: \"kubernetes.io/projected/1530941a-ac98-4f51-9c86-9c6054db9b4e-kube-api-access-kqlzw\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.201863 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.201845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-sys\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.202048 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.202031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-proc\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303589 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-podres\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303777 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqlzw\" (UniqueName: \"kubernetes.io/projected/1530941a-ac98-4f51-9c86-9c6054db9b4e-kube-api-access-kqlzw\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303777 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-sys\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303777 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-podres\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303777 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-proc\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303985 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-proc\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303985 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303810    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-sys\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.303985 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.303859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-lib-modules\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.304087 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.304038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1530941a-ac98-4f51-9c86-9c6054db9b4e-lib-modules\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.311810 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.311779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqlzw\" (UniqueName: \"kubernetes.io/projected/1530941a-ac98-4f51-9c86-9c6054db9b4e-kube-api-access-kqlzw\") pod \"perf-node-gather-daemonset-hf7j4\" (UID: \"1530941a-ac98-4f51-9c86-9c6054db9b4e\") " pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.407707 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.407674    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.697086 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.697006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"]
Sep 16 14:12:57.700719 ip-10-0-130-207 kubenswrapper[2580]: W0916 14:12:57.700688    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1530941a_ac98_4f51_9c86_9c6054db9b4e.slice/crio-05a6b09f45315dc52868d6b953c67cf75aee9806e2d90409e472db497ef0a729 WatchSource:0}: Error finding container 05a6b09f45315dc52868d6b953c67cf75aee9806e2d90409e472db497ef0a729: Status 404 returned error can't find the container with id 05a6b09f45315dc52868d6b953c67cf75aee9806e2d90409e472db497ef0a729
Sep 16 14:12:57.867444 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.867396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4" event={"ID":"1530941a-ac98-4f51-9c86-9c6054db9b4e","Type":"ContainerStarted","Data":"aa1718f59fcce605f80c0b957bae4c47b36723e902f79eb1c71edc54eb26295f"}
Sep 16 14:12:57.867637 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.867469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4" event={"ID":"1530941a-ac98-4f51-9c86-9c6054db9b4e","Type":"ContainerStarted","Data":"05a6b09f45315dc52868d6b953c67cf75aee9806e2d90409e472db497ef0a729"}
Sep 16 14:12:57.868238 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.868210    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:12:57.884195 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:57.884127    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4" podStartSLOduration=0.884104471 podStartE2EDuration="884.104471ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:12:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:12:57.881977361 +0000 UTC m=+2405.565635756" watchObservedRunningTime="2026-09-16 14:12:57.884104471 +0000 UTC m=+2405.567762865"
Sep 16 14:12:58.695441 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:58.695409    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xft82_d56af732-9bed-4dd8-9442-1d0d6ff94734/dns/0.log"
Sep 16 14:12:58.719567 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:58.719532    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xft82_d56af732-9bed-4dd8-9442-1d0d6ff94734/kube-rbac-proxy/0.log"
Sep 16 14:12:58.787517 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:58.787484    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vcjzc_6a28437e-71de-45c2-8064-02e2d89aa324/dns-node-resolver/0.log"
Sep 16 14:12:59.348196 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:12:59.348164    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h4jzc_ed95382d-1fdc-40dd-a557-b20b2e4cb038/node-ca/0.log"
Sep 16 14:13:00.851050 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:00.851026    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bbd697f4-l4hhd_8ce32683-d750-4f31-888c-3db9ef3e21f1/kube-auth-proxy/0.log"
Sep 16 14:13:00.914456 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:00.914419    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-pzqqj_3b109d40-55e2-4176-81f4-28f9197df298/payload-pre-processing/0.log"
Sep 16 14:13:01.907689 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:01.907659    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-snmh9_493adda8-df96-418e-8f5d-b0b611d13187/serve-healthcheck-canary/0.log"
Sep 16 14:13:02.508441 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:02.508376    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvdp2_ad0014ee-7144-46dc-8d4f-4fe57da43308/kube-rbac-proxy/0.log"
Sep 16 14:13:02.530376 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:02.530340    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvdp2_ad0014ee-7144-46dc-8d4f-4fe57da43308/exporter/0.log"
Sep 16 14:13:02.555253 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:02.555223    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wvdp2_ad0014ee-7144-46dc-8d4f-4fe57da43308/extractor/0.log"
Sep 16 14:13:04.889680 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:04.889645    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6t22r/perf-node-gather-daemonset-hf7j4"
Sep 16 14:13:05.002865 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:05.002828    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f96f875d-sxmgg_ebf48b4f-75ba-4e92-95f6-15f1bdfc6e49/manager/0.log"
Sep 16 14:13:05.185960 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:05.185859    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6444576fdb-62lqn_6abfc89e-1c1e-4c68-84dd-f0cbd985a965/manager/0.log"
Sep 16 14:13:05.206243 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:05.206215    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6444576fdb-62lqn_6abfc89e-1c1e-4c68-84dd-f0cbd985a965/copy-manifests/0.log"
Sep 16 14:13:06.704131 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:06.704019    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8878d4c7-q6kzw_3d1d7e2f-251e-473b-afbd-103c162fc02b/manager/0.log"
Sep 16 14:13:06.734826 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:06.734790    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fs2xp_49e50e32-f144-4a8a-bbde-78656e9320e7/openshift-lws-operator/0.log"
Sep 16 14:13:06.993955 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:06.993826    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bbd697f4-l4hhd_8ce32683-d750-4f31-888c-3db9ef3e21f1/kube-auth-proxy/0.log"
Sep 16 14:13:07.040149 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:07.040103    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-pzqqj_3b109d40-55e2-4176-81f4-28f9197df298/payload-pre-processing/0.log"
Sep 16 14:13:09.011469 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:09.011438    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bbd697f4-l4hhd_8ce32683-d750-4f31-888c-3db9ef3e21f1/kube-auth-proxy/0.log"
Sep 16 14:13:09.060931 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:09.060840    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-pzqqj_3b109d40-55e2-4176-81f4-28f9197df298/payload-pre-processing/0.log"
Sep 16 14:13:13.177049 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.177019    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4k6cv_75292479-8bee-4bc8-8263-e77513e1949b/kube-multus/0.log"
Sep 16 14:13:13.238643 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.238608    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlm7v_46a9e2e1-e39a-490d-a9ad-e919b33157dd/kube-multus-additional-cni-plugins/0.log"
Sep 16 14:13:13.260267 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.260243    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlm7v_46a9e2e1-e39a-490d-a9ad-e919b33157dd/egress-router-binary-copy/0.log"
Sep 16 14:13:13.282403 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.282368    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlm7v_46a9e2e1-e39a-490d-a9ad-e919b33157dd/cni-plugins/0.log"
Sep 16 14:13:13.302321 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.302300    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlm7v_46a9e2e1-e39a-490d-a9ad-e919b33157dd/bond-cni-plugin/0.log"
Sep 16 14:13:13.323572 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.323547    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlm7v_46a9e2e1-e39a-490d-a9ad-e919b33157dd/routeoverride-cni/0.log"
Sep 16 14:13:13.345984 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.345958    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlm7v_46a9e2e1-e39a-490d-a9ad-e919b33157dd/whereabouts-cni-bincopy/0.log"
Sep 16 14:13:13.369156 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.369127    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jlm7v_46a9e2e1-e39a-490d-a9ad-e919b33157dd/whereabouts-cni/0.log"
Sep 16 14:13:13.824257 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.824182    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wszjx_6ea9a2a4-11ee-4027-affe-da9400ccd248/network-metrics-daemon/0.log"
Sep 16 14:13:13.840448 ip-10-0-130-207 kubenswrapper[2580]: I0916 14:13:13.840425    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wszjx_6ea9a2a4-11ee-4027-affe-da9400ccd248/kube-rbac-proxy/0.log"